lay out docs skeleton

adding an index

adding data to an index

solr dev tasks

handling results

querying an index

README.md

@@ -19,7 +19,7 @@ Adds support for fulltext search engines like Sphinx and Solr to SilverStripe CM
 
 ## Documentation
 
-See docs/en/index.md
+See [the docs](/docs/en/00_index.md), or for the quick version see [the quick start guide](/docs/en/01_getting_started/11_quick_start.md).
 
 For details of updates, bugfixes, and features, please see the [changelog](CHANGELOG.md).
 

docs/en/00_index.md

@@ -0,0 +1,28 @@
+# Fulltext search documentation index
+
+- Getting started
+    - [Module scope](01_getting_started/10_module_scope.md)
+    - [Quick start guide](01_getting_started/11_quick_start.md)
+- Setup
+    - [Requirements](02_setup/20_requirements.md)
+    - [Installing Solr](02_setup/21_installing_solr.md)
+    - [Installing this module](02_setup/22_installing_the_module.md)
+    - [Solr admin](02_setup/23_solr_admin.md)
+- Configuration
+    - [Creating an index](03_configuration/30_creating_an_index.md)
+    - [Adding data to an index](03_configuration/31_adding_data_to_an_index.md)
+    - [Querying an index](03_configuration/32_querying_the_index.md)
+    - [Running the dev/tasks](03_configuration/33_dev_tasks.md)
+    - [File-based configuration](03_configuration/34_file_based_configuration.md)
+    - [Handling results](03_configuration/35_handling_results.md)
+- Advanced configuration
+    - [Facets](04_advanced_configuration/40_facets.md)
+    - [Using multiple indexes](04_advanced_configuration/41_multiple_indexes.md)
+    - [Synonyms](04_advanced_configuration/42_synonyms.md)
+    - [Spellcheck](04_advanced_configuration/43_spell_check.md)
+    - [Boosting](04_advanced_configuration/44_boosting.md)
+    - [Indexing related objects](04_advanced_configuration/45_indexing_related_objects.md)
+    - [Subsites](04_advanced_configuration/46_subsites.md)
+    - [Adding new fields](04_advanced_configuration/47_adding_new_fields.md)
+- Troubleshooting
+    - [Gotchas](05_troubleshooting/50_common_gotchas.md)

docs/en/01_getting_started/10_module_scope.md

@@ -73,45 +73,9 @@ You can override the default template with a new one at `templates/Layout/Page_r
 ### "Slow" start
 Otherwise, basic usage is a four step process:
 
-1). Define an index in SilverStripe (Note: The specific connector index instance - that's what defines which engine gets used)
-
-```php
-// File: mysite/code/MyIndex.php:
-
-use Page;
-use SilverStripe\FullTextSearch\Solr\SolrIndex;
-
-class MyIndex extends SolrIndex
-{
-    public function init()
-    {
-        $this->addClass(Page::class);
-        $this->addFulltextField('Title');
-        $this->addFulltextField('Content');
-    }
-}
-```
-
-You can also skip listing all searchable fields, and have the index
-figure it out automatically via `addAllFulltextFields()`.
-
-2). Add something to the index (Note: You can also just update an existing document in the CMS. but adding _existing_ objects to the index is connector specific)
-
-```php
-$page = Page::create(['Content' => 'Help me. My house is on fire. This is less than optimal.']);
-$page->write();
-```
-
-Note: There's usually a connector-specific "reindex" task for this.
-
 3). Build a query
 
-```php
-use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
 
-$query = new SearchQuery();
-$query->addSearchTerm('My house is on fire');
-```
 
 4). Apply that query to an index
 

docs/en/03_configuration/30_creating_an_index.md

@@ -0,0 +1,23 @@
+# Creating an index
+
+An index can essentially be considered a database that contains all of your searchable content. By default, it will store everything in a field called `Content`, which is queried to find your search results. To create an index that you can query, you can define it like so:
+
+```php
+use Page;
+use SilverStripe\FullTextSearch\Solr\SolrIndex;
+
+class MyIndex extends SolrIndex
+{
+    public function init()
+    {
+        $this->addClass(Page::class);
+        $this->addFulltextField('Title');
+    }
+}
+```
+
+This will create a new `SolrIndex` called `MyIndex`, and it will store the `Title` field on all `Pages` for searching.
+
+You can also skip listing all searchable fields, and have the index figure it out automatically via `addAllFulltextFields()`. This will add any database fields that are `instanceof DBString` to the index.
+
+Once you've added this file, make sure you run a [Solr configure](./33_dev_tasks.md) to set up your new index.

docs/en/03_configuration/31_adding_data_to_an_index.md

@@ -0,0 +1,30 @@
+# Adding data to an index
+
+Once you have [created your index](./30_creating_an_index.md), you can add data to it in a number of ways.
+
+## Reindex the site
+
+Running the [Solr reindex task](./33_dev_tasks.md) will crawl your site for classes that match those defined on your index, and add the defined fields to the index for searching. This is the most common method used to build the index the first time, or to perform a full rebuild of the index.
+
+## Publish a page in the CMS
+
+Every change, addition or removal of an indexed class instance triggers an index update through a "processor" object. The update is transparently handled through inspecting every executed database query and checking which database tables are involved in it.
+
+A reindex event will trigger when you make a change in the CMS, via `SearchUpdater::handle_manipulation()`, or `ProxyDBExtension::updateProxy()`. This tracks changes to the database, so any alterations will trigger a reindex. In order to minimise delays to those users, the index update is deferred until after the actual request returns to the user, through PHP's `register_shutdown_function()` functionality.
+
+## Manually
+
+If you get desperate, you can create a new page in a build task or something like that:
+
+```php
+use Page;
+
+$page = Page::create(['Content' => 'Help me. My house is on fire. This is less than optimal.']);
+$page->write();
+```
+
+Depending on the size of the index and how much content needs to be updated, it could take a while for your search results to be updated, so try not to panic if your newly-updated page isn't available immediately.
+
+## Queued jobs
+
+If the queuedjobs module is installed, updates are queued up instead of executed in the same request. Queued jobs are usually processed every minute. Large index updates will be batched into multiple queued jobs to ensure a job can run to completion within common constraints, such as memory and execution time limits. You can check the status of jobs in an administrative interface under `admin/queuedjobs/`.

docs/en/03_configuration/32_querying_the_index.md

@@ -0,0 +1,78 @@
+# Querying an index
+
+This is where the magic happens. You will construct the search terms and other parameters required to form a `SearchQuery` object, and pass that into a `SearchIndex` to get results.
+
+## Building a `SearchQuery`
+
+First, you'll need to construct a new `SearchQuery` object:
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+
+$query = SearchQuery::create();
+```
+
+You can then alter the `SearchQuery` with a number of methods:
+
+### `addSearchTerm()`
+
+The simplest - pass through a string to search your index for.
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+
+$query = SearchQuery::create()
+    ->addSearchTerm('fire');
+```
+
+You can also limit this to specific fields by passing an array as the second argument:
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+use Page;
+
+$query = SearchQuery::create()
+    ->addSearchTerm('on fire', [Page::class . '_Title']);
+```
+
+### `addFuzzySearchTerm()`
+
+Pass through a string to search your index for, with "fuzzier" matching - this means that a term like "fishing" would also likely find results containing "fish" or "fisher". Otherwise behaves the same as `addSearchTerm()`.
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+
+$query = SearchQuery::create()
+    ->addFuzzySearchTerm('fire');
+```
+
+### `addClassFilter()`
+
+Only query a specific class in the index, optionally including subclasses.
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+use My\Namespace\PageType\SpecialPage;
+
+$query = SearchQuery::create()
+    ->addClassFilter(SpecialPage::class, false); // only return results from SpecialPages, not subclasses
+```
+
+## Querying an index
+
+Once you have your query constructed, you need to run it against your index.
+
+```php
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+use My\Namespace\Index\MyIndex;
+
+$query = SearchQuery::create()->addSearchTerm('fire');
+$results = singleton(MyIndex::class)->search($query);
+```
+
+The return value of a `search()` call is an object which contains a few properties:
+
+ * `Matches`: `ArrayList` of the current "page" of search results.
+ * `Suggestion`: (optional) Any suggested spelling corrections in the original query notation
+ * `SuggestionNice`: (optional) Any suggested spelling corrections for display (without query notation)
+ * `SuggestionQueryString` (optional) Link to repeat the search with suggested spelling corrections

docs/en/03_configuration/33_dev_tasks.md

@@ -0,0 +1,19 @@
+# Solr dev tasks
+
+There are two dev/tasks that are central to the operation of the module - `Solr_Configure` and `Solr_Reindex`. You can access these through the web, or via CLI. It is often a good idea to run a configure, followed by a reindex, after a code change - for example, after a deployment.
+
+## Solr configure
+
+`dev/tasks/Solr_Configure`
+
+This task will upload configuration to the Solr core, reloading it or creating it as necessary. This should be run after every code change to your indexes, or configuration changes.
+
+## Solr reindex
+
+`dev/tasks/Solr_Reindex`
+
+This task performs a reindex, which adds all the data specified in the index definition into the index store.
+
+If you have the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) installed, then this task will create multiple reindex jobs that are processed asynchronously. Otherwise, it will run in one process. Often, if you are running it via the web, the request will time out. Usually this means the actually process is still running in the background, but it can be alarming to the user.
+
+If instead you run the task via the command line, you will see verbose output as the reindexing progresses.

docs/en/03_configuration/35_handling_results.md

@@ -0,0 +1,47 @@
+# Handling results
+
+In order to render search results, you need to return them from a controller. You can also drive this through a form response through standard SilverStripe forms. In this case we simply assume there's a GET parameter named `q` with a search term present.
+
+```php
+use SilverStripe\CMS\Controllers\ContentController;
+use SilverStripe\Control\HTTPRequest;
+use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
+use My\Namespace\Index\MyIndex;
+
+class PageController extends ContentController
+{
+    private static $allowed_actions = [
+        'search',
+    ];
+
+    public function search(HTTPRequest $request)
+    {
+        $query = SearchQuery::create()->addSearchTerm($request->getVar('q'));
+        return $this->renderWith([
+            'SearchResult' => singleton(MyIndex::class)->search($query)
+        ]);
+    }
+}
+```
+
+In your template (e.g. `Page_results.ss`) you can access the results and loop through them. They're stored in the `$Matches` property of the search return object.
+
+```ss
+<% if $SearchResult.Matches %>
+    <h2>Results for &quot;{$Query}&quot;</h2>
+    <p>Displaying Page $SearchResult.Matches.CurrentPage of $SearchResult.Matches.TotalPages</p>
+    <ol>
+        <% loop $SearchResult.Matches %>
+            <li>
+                <h3><a href="$Link">$Title</a></h3>
+                <p><% if $Abstract %>$Abstract.XML<% else %>$Content.ContextSummary<% end_if %></p>
+            </li>
+        <% end_loop %>
+    </ol>
+<% else %>
+    <p>Sorry, your search query did not return any results.</p>
+<% end_if %>
+```
+
+Please check the [pagination guide](https://docs.silverstripe.org/en/4/developer_guides/templates/how_tos/pagination/)
+in the main SilverStripe documentation to learn how to paginate through search results.