tonyair
/
silverstripe-fulltextsearch
mirror of https://github.com/silverstripe/silverstripe-fulltextsearch
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

restructure to fewer files with anchors

This commit is contained in:
Andrew Aitken-Fincham 2018-05-25 16:14:56 +01:00 committed by Daniel Hensby
parent 51656d94b9
commit 718b1d4e30
No known key found for this signature in database
GPG Key ID: D8DEBC4C8E7BC8B9
28 changed files with 507 additions and 584 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -19,7 +19,7 @@ Adds support for fulltext search engines like Sphinx and Solr to SilverStripe CM
## Documentation
See [the docs](/docs/en/00_index.md), or for the quick version see [the quick start guide](/docs/en/01_getting_started/11_quick_start.md).
See [the docs](/docs/en/00_index.md), or for the quick version see [the quick start guide](/docs/en/01_getting_started.md#quick-start).
For details of updates, bugfixes, and features, please see the [changelog](CHANGELOG.md).

44
docs/en/00_index.md
View File

@ -1,29 +1,29 @@
# Fulltext search documentation index
- Getting started
- [Module scope](01_getting_started/10_module_scope.md)
- [Quick start guide](01_getting_started/11_quick_start.md)
- [Module scope](01_getting_started.md#module-scope)
- [Quick start guide](01_getting_started.md#quick-start)
- Setup
- [Requirements](02_setup/20_requirements.md)
- [Installing Solr](02_setup/21_installing_solr.md)
- [Installing this module](02_setup/22_installing_the_module.md)
- [Solr admin](02_setup/23_solr_admin.md)
- [Requirements](02_setup.md#requirements)
- [Installing Solr](02_setup.md#installing-solr)
- [Installing this module](02_setup.md#installing-the-module)
- [Solr admin](02_setup.md#solr-admin)
- Configuration
- [Creating an index](03_configuration/30_creating_an_index.md)
- [Adding data to an index](03_configuration/31_adding_data_to_an_index.md)
- [Querying an index](03_configuration/32_querying_the_index.md)
- [Running the dev/tasks](03_configuration/33_dev_tasks.md)
- [File-based configuration](03_configuration/34_file_based_configuration.md)
- [Handling results](03_configuration/35_handling_results.md)
- [Solr server parameters](03_configuration.md#solr-server-parameters)
- [Creating an index](03_configuration.md#creating-an-index)
- [Adding data to an index](03_configuration.md#adding-data-to-an-index)
- [Querying an index](03_configuration.md#querying-the-index)
- [Running the dev/tasks](03_configuration.md#dev-tasks)
- [File-based configuration](03_configuration.md#file-based-configuration)
- [Handling results](03_configuration.md#handling-results)
- Advanced configuration
- [Facets](04_advanced_configuration/40_facets.md)
- [Using multiple indexes](04_advanced_configuration/41_multiple_indexes.md)
- [Synonyms](04_advanced_configuration/42_synonyms.md)
- [Spellcheck](04_advanced_configuration/43_spell_check.md)
- [Boosting](04_advanced_configuration/44_boosting.md)
- [Indexing related objects](04_advanced_configuration/45_indexing_related_objects.md)
- [Subsites](04_advanced_configuration/46_subsites.md)
- [Adding new fields](04_advanced_configuration/47_adding_new_fields.md)
- [Facets](04_advanced_configuration.md#facets)
- [Using multiple indexes](04_advanced_configuration.md#multiple-indexes)
- [Synonyms](04_advanced_configuration.md#synonyms)
- [Spellcheck](04_advanced_configuration.md#spell-check)
- [Boosting](04_advanced_configuration.md#boosting)
- [Indexing related objects](04_advanced_configuration.md#indexing-related-objects)
- [Subsites](04_advanced_configuration.md#subsites)
- [Adding new fields](04_advanced_configuration.md#adding-new-fields)
- Troubleshooting
- [Gotchas](05_troubleshooting/50_common_gotchas.md)
- [Test Anchor](05_troubleshooting.md#test-anchor)
- [Gotchas](05_troubleshooting.md#common-gotchas)

10
docs/en/01_getting_started/10_module_scope.md → docs/en/01_getting_started.md
View File

@ -1,4 +1,8 @@
# Introduction
# Getting started
## Module scope
### Introduction
This is a module aimed at adding support for standalone fulltext search engines to SilverStripe.
@ -8,7 +12,7 @@ It contains several layers:
* A connector API, providing common code to allow connecting a fulltext searching engine to the fulltext API
* Some connectors for common fulltext searching engines (currently only [Apache Solr](http://lucene.apache.org/solr/))
## Reasoning
### Reasoning
There are several fulltext search engines that work in a similar manner. They build indexes of denormalized data that
are then searched through using some custom query syntax.
@ -29,3 +33,5 @@ The intent of this module is not to make changing fulltext search engines seamle
common interfaces to fulltext engine functionality, abstracting out common behaviour. However, each connector also
offers its own extensions, and there is some behaviour (such as getting the fulltext search engines installed, configured
and running) that each connector deals with itself, in a way best suited to that search engine's design.
## Quick start

0
docs/en/01_getting_started/11_quick_start.md
View File

60
docs/en/02_setup.md Normal file
View File

@ -0,0 +1,60 @@
# Setup
The fulltextsearch module includes support for connecting to Solr.
It works with Solr in multi-core mode. It needs to be able to update Solr configuration files, and has modes for doing this by direct file access (when Solr shares a server with SilverStripe) and by WebDAV (when it's on a different server).
See the helpful [Solr Tutorial](http://lucene.apache.org/solr/4_5_1/tutorial.html), for more on cores
and querying.
## Requirements
Since Solr is Java based, it requires Java 1.5 or greater installed.
When you're installing it yourself, it also requires a servlet container such as Tomcat, Jetty, or Resin. For
development testing there is a standalone version that comes bundled with Jetty (see [Installing Solr](#installing-solr) below).
See the official [Solr installation docs](http://wiki.apache.org/solr/SolrInstall) for more information.
Note that these requirements are for the Solr server environment, which doesn't have to be the same physical machine as the SilverStripe webhost.
## Installing Solr
### Local installation
If you'll be running Solr on the same machine as your SilverStripe installation, you can use the [silverstripe/fulltextsearch-localsolr module](https://github.com/silverstripe-archive/silverstripe-fulltextsearch-localsolr). This can also be useful as a development dependency. You can bring it in via composer (use `require-dev` if you plan to use install Solr remotely in Production):
```bash
composer require silverstripe/fulltextsearch-localsolr
```
Once installed, start the server via CLI:
```bash
cd fulltextsearch-localsolr/server
java -jar start.jar
```
Then configure Solr to use `file` more with the following configuration in your `app/_config.php`, making sure that the `path` directory is writeable by the user that started the server (above):
```php
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost',
'indexstore' => [
'mode' => 'file',
'path' => BASE_PATH . '/.solr'
]
]);
```
### Remote installation
## Installing the module
## Solr admin

0
docs/en/02_setup/20_requirements.md
View File

0
docs/en/02_setup/21_installing_solr.md
View File

14
docs/en/02_setup/22_installing_the_module.md
View File

@ -1,14 +0,0 @@
# Installing the module
## Disabling automatic configuration
If you have this module installed but do not have a Solr server running, you can disable the database manipulation
hooks that trigger automatic index updates:
```yaml
---
Name: mysitesearch
---
SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater:
enabled: false
```

0
docs/en/02_setup/23_solr_admin.md
View File

340
docs/en/03_configuration.md Normal file
View File

@ -0,0 +1,340 @@
# Configuration
## Solr server parameters
Set these values inside your `app/_config.php` - the defaults are shown below:
```php
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost', // The host or IP address that Solr is listening on
'port' => '8983', // The port Solr is listening on
'path' => '/solr', // The suburl the Solr service is available on
'version' => '4', // Solr server version - currently only 3 and 4 supported
'service' => 'Solr4Service', // The class that provides actual communcation to the Solr server
'extraspath' => BASE_PATH .'/fulltextsearch/conf/solr/4/extras/', // Absolute path to the folder containing templates used for generating the schema and field definitions
'templates' => BASE_PATH . '/fulltextsearch/conf/solr/4/templates/', // Absolute path to the configuration default files, e.g. solrconfig.xml
'indexstore' => [
'mode' => NULL, // [REQUIRED] a classname which implements SolrConfigStore, or 'file' or 'webdav'
'path' => NULL, // [REQUIRED] The (locally accessible) path to write the index configurations to OR The suburl on the Solr host that is set up to accept index configurations via webdav (e.g. BASE_PATH . '/.solr')
'remotepath' => same as 'path' when using 'file' mode, // The path that the Solr server will read the index configurations from
'auth' => NULL, // Webdav only - A username:password pair string to use to auth against the webdav server (e.g. solr:solr)
'port' => '8983' // The port for WebDAV if different from the Solr port
]
]);
```
Note: We recommend to put the `indexstore['path']` directory outside of the webroot. If you place it inside of the webroot (as shown in the example), please ensure its contents are not accessible through the webserver.
This can be achieved by server configuration, or (in most configurations) also by marking the folder as hidden via a "dot" prefix.
### Disabling automatic configuration
If you have this module installed but do not have a Solr server running, you can disable the database manipulation
hooks that trigger automatic index updates:
```yaml
SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater:
enabled: false
```
## Creating an index
An index can essentially be considered a database that contains all of your searchable content. By default, it will store everything in a field called `Content`, which is queried to find your search results. To create an index that you can query, you can define it like so:
```php
use Page;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addFulltextField('Title');
}
}
```
This will create a new `SolrIndex` called `MyIndex`, and it will store the `Title` field on all `Pages` for searching. To index more than one class,
you simply call `addClass()` multiple times. Fields that you add don't have to be present on all classes in the index, they will only apply to a class
if it is present.
```php
use Page;
use SilverStripe\Security\Member;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addClass(Member::class);
$this->addFulltextField('Content'); // only applies to Page class
$this->addFulltextField('FirstName'); // only applies to Member class
}
}
```
You can also skip listing all searchable fields, and have the index figure it out automatically via `addAllFulltextFields()`. This will add any database fields that are `instanceof DBString` to the index. Use this with caution, however, as you may inadvertently return sensitive information - it is often safer to declare your fields explicitly.
Once you've added this file, make sure you run a [Solr configure](#dev_tasks) to set up your new index.
## Adding data to an index
Once you have [created your index](./30_creating_an_index.md), you can add data to it in a number of ways.
### Reindex the site
Running the [Solr reindex task](./33_dev_tasks.md) will crawl your site for classes that match those defined on your index, and add the defined fields to the index for searching. This is the most common method used to build the index the first time, or to perform a full rebuild of the index.
### Publish a page in the CMS
Every change, addition or removal of an indexed class instance triggers an index update through a "processor" object. The update is transparently handled through inspecting every executed database query and checking which database tables are involved in it.
A reindex event will trigger when you make a change in the CMS, via `SearchUpdater::handle_manipulation()`, or `ProxyDBExtension::updateProxy()`. This tracks changes to the database, so any alterations will trigger a reindex. In order to minimise delays to those users, the index update is deferred until after the actual request returns to the user, through PHP's `register_shutdown_function()` functionality.
### Manually
If the situation calls for it, you can add an object to the index directly:
```php
use Page;
$page = Page::create(['Content' => 'Help me. My house is on fire. This is less than optimal.']);
$page->write();
```
Depending on the size of the index and how much content needs to be processed, it could take a while for your search results to be updated, so your newly-updated page may not be available in your search results immediately.
### Queued jobs
If the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) is installed, updates are queued up instead of executed in the same request. Queued jobs are usually processed every minute. Large index updates will be batched into multiple queued jobs to ensure a job can run to completion within common constraints, such as memory and execution time limits. You can check the status of jobs in an administrative interface under `admin/queuedjobs/`.
### Excluding draft content
By default, the `SearchUpdater` class indexes all available "variant states", so in the case of the `Versioned` extension, both "draft" and "live".
For most cases, you'll want to exclude draft content from your search results.
You can either prevent the draft content from being indexed in the first place, by adding the following to your `SearchIndex::init()` method:
```php
use Page;
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
use SilverStripe\Versioned\Versioned;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addFulltextField('Title');
$this->excludeVariantState([SearchVariantVersioned::class => Versioned::DRAFT]);
}
}
```
Alternatively, you can index draft content, but simply exclude it from searches. This can be handy to preview search results on unpublished content, in case a CMS author is logged in. Before constructing your `SearchQuery`, conditionally switch to the "live" stage:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\Security\Permission;
use SilverStripe\Versioned\Versioned;
if (!Permission::check('CMS_ACCESS_CMSMain')) {
Versioned::set_stage(Versioned::LIVE);
}
$query = SearchQuery::create();
// ...
```
## Querying an index
This is where the magic happens. You will construct the search terms and other parameters required to form a `SearchQuery` object, and pass that into a `SearchIndex` to get results.
### Building a `SearchQuery`
First, you'll need to construct a new `SearchQuery` object:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create();
```
You can then alter the `SearchQuery` with a number of methods:
#### `addSearchTerm()`
The simplest - pass through a string to search your index for.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create()
->addSearchTerm('fire');
```
You can also limit this to specific fields by passing an array as the second argument:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use Page;
$query = SearchQuery::create()
->addSearchTerm('on fire', [Page::class . '_Title']);
```
#### `addFuzzySearchTerm()`
Pass through a string to search your index for, with "fuzzier" matching - this means that a term like "fishing" would also likely find results containing "fish" or "fisher". Otherwise behaves the same as `addSearchTerm()`.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create()
->addFuzzySearchTerm('fire');
```
#### `addClassFilter()`
Only query a specific class in the index, optionally including subclasses.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\PageType\SpecialPage;
$query = SearchQuery::create()
->addClassFilter(SpecialPage::class, false); // only return results from SpecialPages, not subclasses
```
#### Searching value ranges
Most values can be expressed as ranges, most commonly dates or numbers. To search for a range of values rather than an exact match,
use the `SearchQuery_Range` class. The range can include bounds on both sides, or stay open-ended by simply leaving the argument blank.
It takes arguments in the form of `SearchQuery_Range::create($start, $end))`:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery_Range;
use My\Namespace\Index\MyIndex;
use Page;
$query = SearchQuery::create()
->addSearchTerm('fire')
// Only include documents edited in 2011 or earlier
->addFilter(Page::class . '_LastEdited', SearchQuery_Range::create(null, '2011-12-31T23:59:59Z'));
$results = singleton(MyIndex::class)->search($query);
```
Note: At the moment, the date format is specific to the search implementation.
#### Searching for empty or existing values
Since there's a type conversion between the SilverStripe database, object properties
and the search index persistence, it's often not clear which condition is searched for.
Should it equal an empty string, or only match if the field wasn't indexed at all?
The `SearchQuery` API has the concept of a "missing" and "present" field value for this:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
use Page;
$query = SearchQuery::create()
->addSearchTerm('fire');
// Needs a value, although it can be false
->addFilter(Page::class . '_ShowInMenus', SearchQuery::$present);
$results = singleton(MyIndex::class)->search($query);
```
### Querying an index
Once you have your query constructed, you need to run it against your index.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
$query = SearchQuery::create()->addSearchTerm('fire');
$results = singleton(MyIndex::class)->search($query);
```
The return value of a `search()` call is an object which contains a few properties:
* `Matches`: `ArrayList` of the current "page" of search results.
* `Suggestion`: (optional) Any suggested spelling corrections in the original query notation
* `SuggestionNice`: (optional) Any suggested spelling corrections for display (without query notation)
* `SuggestionQueryString` (optional) Link to repeat the search with suggested spelling corrections
## Solr dev tasks
There are two dev/tasks that are central to the operation of the module - `Solr_Configure` and `Solr_Reindex`. You can access these through the web, or via CLI. Running via the web will return "quiet" output by default, but you can increase verbosity by adding `?verbose=1` to the `dev/tasks` URL; CLI will return verbose output by default.
It is often a good idea to run a configure, followed by a reindex, after a code change - for example, after a deployment.
### Solr configure
`dev/tasks/Solr_Configure`
This task will upload configuration to the Solr core, reloading it or creating it as necessary. This should be run after every code change to your indexes, or configuration changes.
### Solr reindex
`dev/tasks/Solr_Reindex`
This task performs a reindex, which adds all the data specified in the index definition into the index store.
If you have the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) installed, then this task will create multiple reindex jobs that are processed asynchronously; unless you are in `dev` mode, in which case the index will be processed immediately (see [processor.yml](/_config/processor.yml)). Otherwise, it will run in one process. Often, if you are running it via the web, the request will time out. Usually this means the actually process is still running in the background, but it can be alarming to the user, so bear that in mind.
## File-based configuration
## Handling results
In order to render search results, you need to return them from a controller. You can also drive this through a form response through standard SilverStripe forms. In this case we simply assume there's a GET parameter named `q` with a search term present.
```php
use SilverStripe\CMS\Controllers\ContentController;
use SilverStripe\Control\HTTPRequest;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
class PageController extends ContentController
{
private static $allowed_actions = [
'search',
];
public function search(HTTPRequest $request)
{
$query = SearchQuery::create()->addSearchTerm($request->getVar('q'));
return $this->renderWith([
'SearchResult' => singleton(MyIndex::class)->search($query)
]);
}
}
```
In your template (e.g. `Page_results.ss`) you can access the results and loop through them. They're stored in the `$Matches` property of the search return object.
```ss
<% if $SearchResult.Matches %>
<h2>Results for &quot;{$Query}&quot;</h2>
<p>Displaying Page $SearchResult.Matches.CurrentPage of $SearchResult.Matches.TotalPages</p>
<ol>
<% loop $SearchResult.Matches %>
<li>
<h3><a href="$Link">$Title</a></h3>
<p><% if $Abstract %>$Abstract.XML<% else %>$Content.ContextSummary<% end_if %></p>
</li>
<% end_loop %>
</ol>
<% else %>
<p>Sorry, your search query did not return any results.</p>
<% end_if %>
```
Please check the [pagination guide](https://docs.silverstripe.org/en/4/developer_guides/templates/how_tos/pagination/)
in the main SilverStripe documentation to learn how to paginate through search results.

42
docs/en/03_configuration/30_creating_an_index.md
View File

@ -1,42 +0,0 @@
# Creating an index
An index can essentially be considered a database that contains all of your searchable content. By default, it will store everything in a field called `Content`, which is queried to find your search results. To create an index that you can query, you can define it like so:
```php
use Page;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addFulltextField('Title');
}
}
```
This will create a new `SolrIndex` called `MyIndex`, and it will store the `Title` field on all `Pages` for searching. To index more than one class,
you simply call `addClass()` multiple times. Fields that you add don't have to be present on all classes in the index, they will only apply to a class
if it is present.
```php
use Page;
use SilverStripe\Security\Member;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addClass(Member::class);
$this->addFulltextField('Content'); // only applies to Page class
$this->addFulltextField('FirstName'); // only applies to Member class
}
}
```
You can also skip listing all searchable fields, and have the index figure it out automatically via `addAllFulltextFields()`. This will add any database fields that are `instanceof DBString` to the index. Use this with caution, however, as you may inadvertently return sensitive information - it is often safer to declare your fields explicitly.
Once you've added this file, make sure you run a [Solr configure](./33_dev_tasks.md) to set up your new index.

68
docs/en/03_configuration/31_adding_data_to_an_index.md
View File

@ -1,68 +0,0 @@
# Adding data to an index
Once you have [created your index](./30_creating_an_index.md), you can add data to it in a number of ways.
## Reindex the site
Running the [Solr reindex task](./33_dev_tasks.md) will crawl your site for classes that match those defined on your index, and add the defined fields to the index for searching. This is the most common method used to build the index the first time, or to perform a full rebuild of the index.
## Publish a page in the CMS
Every change, addition or removal of an indexed class instance triggers an index update through a "processor" object. The update is transparently handled through inspecting every executed database query and checking which database tables are involved in it.
A reindex event will trigger when you make a change in the CMS, via `SearchUpdater::handle_manipulation()`, or `ProxyDBExtension::updateProxy()`. This tracks changes to the database, so any alterations will trigger a reindex. In order to minimise delays to those users, the index update is deferred until after the actual request returns to the user, through PHP's `register_shutdown_function()` functionality.
## Manually
If the situation calls for it, you can add an object to the index directly:
```php
use Page;
$page = Page::create(['Content' => 'Help me. My house is on fire. This is less than optimal.']);
$page->write();
```
Depending on the size of the index and how much content needs to be processed, it could take a while for your search results to be updated, so your newly-updated page may not be available in your search results immediately.
## Queued jobs
If the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) is installed, updates are queued up instead of executed in the same request. Queued jobs are usually processed every minute. Large index updates will be batched into multiple queued jobs to ensure a job can run to completion within common constraints, such as memory and execution time limits. You can check the status of jobs in an administrative interface under `admin/queuedjobs/`.
### Excluding draft content
By default, the `SearchUpdater` class indexes all available "variant states", so in the case of the `Versioned` extension, both "draft" and "live".
For most cases, you'll want to exclude draft content from your search results.
You can either prevent the draft content from being indexed in the first place, by adding the following to your `SearchIndex::init()` method:
```php
use Page;
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
use SilverStripe\Versioned\Versioned;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addFulltextField('Title');
$this->excludeVariantState([SearchVariantVersioned::class => Versioned::DRAFT]);
}
}
```
Alternatively, you can index draft content, but simply exclude it from searches. This can be handy to preview search results on unpublished content, in case a CMS author is logged in. Before constructing your `SearchQuery`, conditionally switch to the "live" stage:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\Security\Permission;
use SilverStripe\Versioned\Versioned;
if (!Permission::check('CMS_ACCESS_CMSMain')) {
Versioned::set_stage(Versioned::LIVE);
}
$query = SearchQuery::create();
// ...
```

118
docs/en/03_configuration/32_querying_the_index.md
View File

@ -1,118 +0,0 @@
# Querying an index
This is where the magic happens. You will construct the search terms and other parameters required to form a `SearchQuery` object, and pass that into a `SearchIndex` to get results.
## Building a `SearchQuery`
First, you'll need to construct a new `SearchQuery` object:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create();
```
You can then alter the `SearchQuery` with a number of methods:
### `addSearchTerm()`
The simplest - pass through a string to search your index for.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create()
->addSearchTerm('fire');
```
You can also limit this to specific fields by passing an array as the second argument:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use Page;
$query = SearchQuery::create()
->addSearchTerm('on fire', [Page::class . '_Title']);
```
### `addFuzzySearchTerm()`
Pass through a string to search your index for, with "fuzzier" matching - this means that a term like "fishing" would also likely find results containing "fish" or "fisher". Otherwise behaves the same as `addSearchTerm()`.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = SearchQuery::create()
->addFuzzySearchTerm('fire');
```
### `addClassFilter()`
Only query a specific class in the index, optionally including subclasses.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\PageType\SpecialPage;
$query = SearchQuery::create()
->addClassFilter(SpecialPage::class, false); // only return results from SpecialPages, not subclasses
```
### Searching value ranges
Most values can be expressed as ranges, most commonly dates or numbers. To search for a range of values rather than an exact match,
use the `SearchQuery_Range` class. The range can include bounds on both sides, or stay open-ended by simply leaving the argument blank.
It takes arguments in the form of `SearchQuery_Range::create($start, $end))`:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery_Range;
use My\Namespace\Index\MyIndex;
use Page;
$query = SearchQuery::create()
->addSearchTerm('fire')
// Only include documents edited in 2011 or earlier
->addFilter(Page::class . '_LastEdited', SearchQuery_Range::create(null, '2011-12-31T23:59:59Z'));
$results = singleton(MyIndex::class)->search($query);
```
Note: At the moment, the date format is specific to the search implementation.
### Searching for empty or existing values
Since there's a type conversion between the SilverStripe database, object properties
and the search index persistence, it's often not clear which condition is searched for.
Should it equal an empty string, or only match if the field wasn't indexed at all?
The `SearchQuery` API has the concept of a "missing" and "present" field value for this:
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
use Page;
$query = SearchQuery::create()
->addSearchTerm('fire');
// Needs a value, although it can be false
->addFilter(Page::class . '_ShowInMenus', SearchQuery::$present);
$results = singleton(MyIndex::class)->search($query);
```
## Querying an index
Once you have your query constructed, you need to run it against your index.
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
$query = SearchQuery::create()->addSearchTerm('fire');
$results = singleton(MyIndex::class)->search($query);
```
The return value of a `search()` call is an object which contains a few properties:
* `Matches`: `ArrayList` of the current "page" of search results.
* `Suggestion`: (optional) Any suggested spelling corrections in the original query notation
* `SuggestionNice`: (optional) Any suggested spelling corrections for display (without query notation)
* `SuggestionQueryString` (optional) Link to repeat the search with suggested spelling corrections

19
docs/en/03_configuration/33_dev_tasks.md
View File

@ -1,19 +0,0 @@
# Solr dev tasks
There are two dev/tasks that are central to the operation of the module - `Solr_Configure` and `Solr_Reindex`. You can access these through the web, or via CLI. Running via the web will return "quiet" output by default, but you can increase verbosity by adding `?verbose=1` to the `dev/tasks` URL; CLI will return verbose output by default.
It is often a good idea to run a configure, followed by a reindex, after a code change - for example, after a deployment.
## Solr configure
`dev/tasks/Solr_Configure`
This task will upload configuration to the Solr core, reloading it or creating it as necessary. This should be run after every code change to your indexes, or configuration changes.
## Solr reindex
`dev/tasks/Solr_Reindex`
This task performs a reindex, which adds all the data specified in the index definition into the index store.
If you have the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) installed, then this task will create multiple reindex jobs that are processed asynchronously; unless you are in `dev` mode, in which case the index will be processed immediately (see [processor.yml](/_config/processor.yml)). Otherwise, it will run in one process. Often, if you are running it via the web, the request will time out. Usually this means the actually process is still running in the background, but it can be alarming to the user, so bear that in mind.

0
docs/en/03_configuration/34_file_based_configuration.md
View File

47
docs/en/03_configuration/35_handling_results.md
View File

@ -1,47 +0,0 @@
# Handling results
In order to render search results, you need to return them from a controller. You can also drive this through a form response through standard SilverStripe forms. In this case we simply assume there's a GET parameter named `q` with a search term present.
```php
use SilverStripe\CMS\Controllers\ContentController;
use SilverStripe\Control\HTTPRequest;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use My\Namespace\Index\MyIndex;
class PageController extends ContentController
{
private static $allowed_actions = [
'search',
];
public function search(HTTPRequest $request)
{
$query = SearchQuery::create()->addSearchTerm($request->getVar('q'));
return $this->renderWith([
'SearchResult' => singleton(MyIndex::class)->search($query)
]);
}
}
```
In your template (e.g. `Page_results.ss`) you can access the results and loop through them. They're stored in the `$Matches` property of the search return object.
```ss
<% if $SearchResult.Matches %>
<h2>Results for &quot;{$Query}&quot;</h2>
<p>Displaying Page $SearchResult.Matches.CurrentPage of $SearchResult.Matches.TotalPages</p>
<ol>
<% loop $SearchResult.Matches %>
<li>
<h3><a href="$Link">$Title</a></h3>
<p><% if $Abstract %>$Abstract.XML<% else %>$Content.ContextSummary<% end_if %></p>
</li>
<% end_loop %>
</ol>
<% else %>
<p>Sorry, your search query did not return any results.</p>
<% end_if %>
```
Please check the [pagination guide](https://docs.silverstripe.org/en/4/developer_guides/templates/how_tos/pagination/)
in the main SilverStripe documentation to learn how to paginate through search results.

75
docs/en/04_advanced_configuration.md Normal file
View File

@ -0,0 +1,75 @@
# Advanced configuration
## Facets
## Multiple indexes
Multiple indexes can be created and searched independently, but if you wish to override an existing
index with another, you can use the `$hide_ancestor` config.
```php
use SilverStripe\Assets\File;
use My\Namespace\Index\MyIndex;
class MyReplacementIndex extends MyIndex
{
private static $hide_ancestor = MyIndex::class;
public function init()
{
parent::init();
$this->addClass(File::class);
$this->addFulltextField('Title');
}
}
```
You can also filter all indexes globally to a set of pre-defined classes if you wish to
prevent any unknown indexes from being automatically included.
```yaml
SilverStripe\FullTextSearch\Search\FullTextSearch:
indexes:
- MyReplacementIndex
- CoreSearchIndex
```
## Synonyms
## Spell check
## Boosting/Weighting
Results aren't all created equal. Matches in some fields are more important
than others; for example, a page `Title` might be considered more relevant to the user than terms in the `Content` field.
To account for this, a "weighting" (or "boosting") factor can be applied to each searched field. The default value is `1.0`, anything below that will decrease the relevance, anything above increases it.
To adjust the relative values, pass them in as the third argument to your `addSearchTerm()` call:
```php
use My\Namespace\Index\MyIndex;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use Page;
$query = SearchQuery::create()
->addSearchTerm(
'fire',
null, // don't limit the classes to search
[
Page::class . '_Title' => 1.5,
Page::class . '_Content' => 1.0,
Page::class . '_SecretParagraph' => 0.1,
]
);
$results = singleton(MyIndex::class)->search($query);
```
This will ensure that `Title` is given higher priority for matches than `Content`, which is well above `SecretParagraph`.
## Indexing related objects
## Subsites
## Adding new fields

0
docs/en/04_advanced_configuration/40_facets.md
View File

32
docs/en/04_advanced_configuration/41_multiple_indexes.md
View File

@ -1,32 +0,0 @@
# Using multiple indexes
Multiple indexes can be created and searched independently, but if you wish to override an existing
index with another, you can use the `$hide_ancestor` config.
```php
use SilverStripe\Assets\File;
use My\Namespace\Index\MyIndex;
class MyReplacementIndex extends MyIndex
{
private static $hide_ancestor = MyIndex::class;
public function init()
{
parent::init();
$this->addClass(File::class);
$this->addFulltextField('Title');
}
}
```
You can also filter all indexes globally to a set of pre-defined classes if you wish to
prevent any unknown indexes from being automatically included.
```yaml
SilverStripe\FullTextSearch\Search\FullTextSearch:
indexes:
- MyReplacementIndex
- CoreSearchIndex
```

0
docs/en/04_advanced_configuration/42_synonyms.md
View File

0
docs/en/04_advanced_configuration/43_spell_check.md
View File

28
docs/en/04_advanced_configuration/44_boosting.md
View File

@ -1,28 +0,0 @@
# Boosting/Weighting
Results aren't all created equal. Matches in some fields are more important
than others; for example, a page `Title` might be considered more relevant to the user than terms in the `Content` field.
To account for this, a "weighting" (or "boosting") factor can be applied to each searched field. The default value is `1.0`, anything below that will decrease the relevance, anything above increases it.
To adjust the relative values, pass them in as the third argument to your `addSearchTerm()` call:
```php
use My\Namespace\Index\MyIndex;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use Page;
$query = SearchQuery::create()
->addSearchTerm(
'fire',
null, // don't limit the classes to search
[
Page::class . '_Title' => 1.5,
Page::class . '_Content' => 1.0,
Page::class . '_SecretParagraph' => 0.1,
]
);
$results = singleton(MyIndex::class)->search($query);
```
This will ensure that `Title` is given higher priority for matches than `Content`, which is well above `SecretParagraph`.

0
docs/en/04_advanced_configuration/45_indexing_related_objects.md
View File

0
docs/en/04_advanced_configuration/46_subsites.md
View File

0
docs/en/04_advanced_configuration/47_adding_new_fields.md
View File

111
docs/en/05_troubleshooting.md
View File

@ -1,112 +1,3 @@
# Troubleshooting
## Common Gotchas
Oooh I gotcha
Here's some whitespace
## Test anchor
I'm on a boat
## Common gotchas

0
docs/en/05_troubleshooting/50_common_gotchas.md
View File

81
docs/en/Solr.md
View File

@ -1,90 +1,9 @@
# Solr connector for SilverStripe fulltextsearch module
## Introduction
The fulltextsearch module includes support for connecting to Solr.
It works with Solr in multi-core mode. It needs to be able to update Solr configuration files, and has modes for
doing this by direct file access (when Solr shares a server with SilverStripe) and by WebDAV (when it's on a different
server).
See the helpful [Solr Tutorial](http://lucene.apache.org/solr/4_5_1/tutorial.html), for more on cores
and querying.
## Requirements
Since Solr is Java based, it requires Java 1.5 or greater installed.
When you're installing it yourself, it also requires a servlet container such as Tomcat, Jetty, or Resin. For
development testing there is a standalone version that comes bundled with Jetty (see below).
See the official [Solr installation docs](http://wiki.apache.org/solr/SolrInstall) for more information.
Note that these requirements are for the Solr server environment, which doesn't have to be the same physical machine
as the SilverStripe webhost.
## Installation (Local)
### Get the Solr server
```
composer require silverstripe/fulltextsearch-localsolr
```
### Start the server (via CLI, in a separate terminal window or background process)
```
cd fulltextsearch-localsolr/server/
java -jar start.jar
```
### Configure the fulltextsearch Solr component to use the local server
Configure Solr in file mode. The 'path' directory has to be writeable
by the user the Solr search server is started with (see below).
```php
// File: mysite/_config.php:
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost',
'indexstore' => [
'mode' => 'file',
'path' => BASE_PATH . '/.solr'
]
]);
```
All possible parameters incl optional ones with example values:
```php
// File: mysite/_config.php:
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost', // default: localhost | The host or IP Solr is listening on
'port' => '8983', // default: 8983 | The port Solr is listening on
'path' => '/solr', // default: /solr | The suburl the solr service is available on
'version' => '4', // default: 4 | Solr server version - currently only 3 and 4 supported
'service' => 'Solr4Service', // default: depends on version, Solr3Service for 3, Solr4Service for 4 | the class that provides actual communcation to the Solr server
'extraspath' => BASE_PATH .'/fulltextsearch/conf/solr/4/extras/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/extras/ | Absolute path to the folder containing templates which are used for generating the schema and field definitions.
'templates' => BASE_PATH . '/fulltextsearch/conf/solr/4/templates/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/templates/ | Absolute path to the configuration default files, e.g. solrconfig.xml
'indexstore' => [
'mode' => 'file', // a classname which implements SolrConfigStore, or 'file' or 'webdav'
'path' => BASE_PATH . '/.solr', // The (locally accessible) path to write the index configurations to OR The suburl on the solr host that is set up to accept index configurations via webdav
'remotepath' => '/opt/solr/config', // default (file mode only): same as 'path' above | The path that the Solr server will read the index configurations from
'auth' => 'solr:solr', // default: none | Webdav only - A username:password pair string to use to auth against the webdav server
'port' => '80' // default: same as solr port | The port for WebDAV if different from the Solr port
]
]);
```
Note: We recommend to put the `indexstore.path` directory outside of the webroot.
If you place it inside of the webroot (as shown in the example),
please ensure its contents are not accessible through the webserver.
This can be achieved by server configuration, or (in most configurations)
also by marking the folder as hidden via a "dot" prefix.
## Configuration