DOCS Update namespaces and code formatting in documentation

This commit is contained in:
Robbie Averill 2017-11-21 10:51:32 +13:00
parent fbc4c0e81d
commit 74db46f5d7
3 changed files with 526 additions and 340 deletions

View File

@ -29,7 +29,7 @@ For details of updates, bugfixes, and features, please see the [changelog](CHANG
used at query time for most of the same use cases
* Fix field referencing in queries. Should be able to do `$query->search('Text', 'Content')`, not
`$query->search('Text', 'SiteTree_Content')` like you have to do now
`$query->search('Text', SiteTree::class . '_Content')` like you have to do now
- Make sure that when field exists in multiple classes, searching against bare fields searches all of them

View File

@ -27,49 +27,58 @@ as the SilverStripe webhost.
### Get the Solr server
composer require silverstripe/fulltextsearch-localsolr 4.5.1.x-dev
```
composer require silverstripe/fulltextsearch-localsolr
```
### Start the server (via CLI, in a separate terminal window or background process)
cd fulltextsearch-localsolr/server/
java -jar start.jar
```
cd fulltextsearch-localsolr/server/
java -jar start.jar
```
### Configure the fulltextsearch Solr component to use the local server
Configure Solr in file mode. The 'path' directory has to be writeable
by the user the Solr search server is started with (see below).
// File: mysite/_config.php:
<?php
Solr::configure_server(array(
'host' => 'localhost',
'indexstore' => array(
'mode' => 'file',
'path' => BASE_PATH . '/.solr'
)
));
```php
// File: mysite/_config.php:
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost',
'indexstore' => [
'mode' => 'file',
'path' => BASE_PATH . '/.solr'
]
]);
```
All possible parameters incl optional ones with example values:
// File: mysite/_config.php:
<?php
Solr::configure_server(array(
'host' => 'localhost', // default: localhost | The host or IP Solr is listening on
'port' => '8983', // default: 8983 | The port Solr is listening on
'path' => '/solr', // default: /solr | The suburl the solr service is available on
'version' => '4', // default: 4 | Solr server version - currently only 3 and 4 supported
'service' => 'Solr4Service', // default: depends on version, Solr3Service for 3, Solr4Service for 4 | the class that provides actual communcation to the Solr server
'extraspath' => BASE_PATH .'/fulltextsearch/conf/solr/4/extras/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/extras/ | Absolute path to the folder containing templates which are used for generating the schema and field definitions.
'templates' => BASE_PATH . '/fulltextsearch/conf/solr/4/templates/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/templates/ | Absolute path to the configuration default files, e.g. solrconfig.xml
'indexstore' => array(
'mode' => 'file', // a classname which implements SolrConfigStore, or 'file' or 'webdav'
'path' => BASE_PATH . '/.solr', // The (locally accessible) path to write the index configurations to OR The suburl on the solr host that is set up to accept index configurations via webdav
'remotepath' => '/opt/solr/config', // default (file mode only): same as 'path' above | The path that the Solr server will read the index configurations from
'auth' => 'solr:solr', // default: none | Webdav only - A username:password pair string to use to auth against the webdav server
'port' => '80' // default: same as solr port | The port for WebDAV if different from the Solr port
)
));
```php
// File: mysite/_config.php:
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
'host' => 'localhost', // default: localhost | The host or IP Solr is listening on
'port' => '8983', // default: 8983 | The port Solr is listening on
'path' => '/solr', // default: /solr | The suburl the solr service is available on
'version' => '4', // default: 4 | Solr server version - currently only 3 and 4 supported
'service' => 'Solr4Service', // default: depends on version, Solr3Service for 3, Solr4Service for 4 | the class that provides actual communcation to the Solr server
'extraspath' => BASE_PATH .'/fulltextsearch/conf/solr/4/extras/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/extras/ | Absolute path to the folder containing templates which are used for generating the schema and field definitions.
'templates' => BASE_PATH . '/fulltextsearch/conf/solr/4/templates/', // default: <basefolder>/fulltextsearch/conf/solr/{version}/templates/ | Absolute path to the configuration default files, e.g. solrconfig.xml
'indexstore' => [
'mode' => 'file', // a classname which implements SolrConfigStore, or 'file' or 'webdav'
'path' => BASE_PATH . '/.solr', // The (locally accessible) path to write the index configurations to OR The suburl on the solr host that is set up to accept index configurations via webdav
'remotepath' => '/opt/solr/config', // default (file mode only): same as 'path' above | The path that the Solr server will read the index configurations from
'auth' => 'solr:solr', // default: none | Webdav only - A username:password pair string to use to auth against the webdav server
'port' => '80' // default: same as solr port | The port for WebDAV if different from the Solr port
]
]);
```
Note: We recommend to put the `indexstore.path` directory outside of the webroot.
If you place it inside of the webroot (as shown in the example),
@ -81,30 +90,37 @@ also by marking the folder as hidden via a "dot" prefix.
### Create an index
// File: mysite/code/MyIndex.php:
<?php
class MyIndex extends SolrIndex {
function init() {
$this->addClass('Page');
$this->addAllFulltextFields();
}
}
```php
// File: mysite/code/MyIndex.php:
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addAllFulltextFields();
}
}
```
### Create the index schema
The PHP-based index definition is an abstraction layer for the actual Solr XML configuration.
In order to create or update it, you need to run the `Solr_Configure` task.
sake dev/tasks/Solr_Configure
```
vendor/bin/sake dev/tasks/Solr_Configure
```
Based on the sample configuration above, this command will do the following:
- Create a `<BASE_PATH>/.solr/MyIndex` folder
- Copy configuration files from `fulltextsearch/conf/extras/` to `<BASE_PATH>/.solr/MyIndex/conf`
- Copy configuration files from `vendor/silverstripe/fulltextsearch/conf/extras/` to `<BASE_PATH>/.solr/MyIndex/conf`
- Generate a `schema.xml`, and place it it in `<BASE_PATH>/.solr/MyIndex/conf`
If you call the task with an existing index folder,
it will overwrite all files from their default locations,
it will overwrite all files from their default locations,
regenerate the `schema.xml`, and ask Solr to reload the configuration.
You can use the same command for updating an existing schema,
@ -115,7 +131,9 @@ which will automatically apply without requiring a Solr server restart.
After configuring Solr, you have the option to add your existing
content to its indices. Run the following command:
sake dev/tasks/Solr_Reindex
```
vendor/bin/sake dev/tasks/Solr_Reindex
```
This will delete and rebuild all indices. Depending on your data,
this can take anywhere from minutes to hours.
@ -135,13 +153,12 @@ as crontasks, or via separate processes initiated by the current request.
Internally groups of records are grouped into sizes of 200. You can configure this
group sizing by using the `Solr_Reindex.recordsPerRequest` config.
```yaml
SilverStripe\FullTextSearch\Solr\Tasks\Solr_Reindex:
recordsPerRequest: 150
```
:::yaml
Solr_Reindex:
recordsPerRequest: 150
Note: The Solr indexes will be stored as binary files inside your SilverStripe project.
Note: The Solr indexes will be stored as binary files inside your SilverStripe project.
You can also copy the `thirdparty/` solr directory somewhere else,
just set the `path` value in `mysite/_config.php` to point to the new location.
@ -163,12 +180,17 @@ By default, these files are copied from the `fulltextsearch/conf/extras/`
directory over to the new index location. In order to use your own files,
copy these files into a location of your choosing (for example `mysite/data/solr/`),
and tell Solr to use this folder with the `extraspath` configuration setting.
// mysite/_config.php
Solr::configure_server(array(
// ...
'extraspath' => Director::baseFolder() . '/mysite/data/solr/',
));
```php
// mysite/_config.php
use SilverStripe\Control\Director;
use SilverStripe\FullTextSearch\Solr\Solr;
Solr::configure_server([
// ...
'extraspath' => Director::baseFolder() . '/mysite/data/solr/',
]);
```
Please run the `Solr_Configure` task for the changes to take effect.
@ -185,16 +207,22 @@ by overloading the template responsible for it: `types.ss`.
In the following example, we read out type definitions
from a new file `mysite/solr/templates/types.ss` instead:
<?php
class MyIndex extends SolrIndex {
function getTypes() {
return $this->renderWith(Director::baseFolder() . '/mysite/solr/templates/types.ss');
}
}
```php
use SilverStripe\Control\Director;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function getTypes()
{
return $this->renderWith(Director::baseFolder() . '/mysite/solr/templates/types.ss');
}
}
```
#### Searching for words containing numbers
By default, the fulltextmodule is configured to split words containing numbers into multiple tokens. For example, the word “A1” would be interpreted as “A” “1”; since “a” is a common stopword, the term “A1” will be excluded from search.
By default, the fulltextmodule is configured to split words containing numbers into multiple tokens. For example, the word "A1" would be interpreted as "A" "1"; since "a" is a common stopword, the term "A1" will be excluded from search.
To allow searches on words containing numeric tokens, you'll need to update your overloaded template to change the behaviour of the WordDelimiterFilterFactory. Each instance of `<filter class="solr.WordDelimiterFilterFactory">` needs to include the following attributes and values:
@ -205,15 +233,19 @@ Update your index to point to your overloaded template using the method describe
#### Searching for macrons and other Unicode characters
The “ASCIIFoldingFilterFactory” filter converts alphabetic, numeric, and symbolic Unicode characters which are not in the Basic Latin Unicode block (the first 127 ASCII characters) to their ASCII equivalents, if one exists.
The "ASCIIFoldingFilterFactory" filter converts alphabetic, numeric, and symbolic Unicode characters which are not in the Basic Latin Unicode block (the first 127 ASCII characters) to their ASCII equivalents, if one exists.
Find the fields in your overloaded `types.ss` that you want to enable this behaviour in. EG:
<fieldType name="htmltext" class="solr.TextField" ... >
```xml
<fieldType name="htmltext" class="solr.TextField" ... >
```
Add the following to both its index analyzer and query analyzer records.
<filter class="solr.ASCIIFoldingFilterFactory"/>
```xml
<filter class="solr.ASCIIFoldingFilterFactory"/>
```
Update your index to point to your overloaded template using the method described above.
@ -225,55 +257,68 @@ spell checking data is collected from all fulltext fields
(everything you added through `SolrIndex->addFulltextField()`).
The values of these fields are collected in a special `_text` field.
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$params = array('spellcheck' => 'true', 'spellcheck.collate' => 'true');
$results = $index->search($query, -1, -1, $params);
$results->spellcheck
```php
use SilverStripe\FullTextSearch\Search\Queries;
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$params = [
'spellcheck' => 'true',
'spellcheck.collate' => 'true',
];
$results = $index->search($query, -1, -1, $params);
$results->spellcheck
```
The built-in `_text` data is better than nothing, but also has some problems:
Its heavily processed, for example by stemming filters which butcher words.
So misspelling "Govnernance" will suggest "govern" rather than "Governance".
This can be fixed by aggregating spell checking data in a separate
<?php
class MyIndex extends SolrIndex {
```php
use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
function init() {
// ...
$this->addCopyField('SiteTree_Title', 'spellcheckData');
$this->addCopyField('DMSDocument_Title', 'spellcheckData');
$this->addCopyField('SiteTree_Content', 'spellcheckData');
$this->addCopyField('DMSDocument_Content', 'spellcheckData');
}
class MyIndex extends SolrIndex
{
public function init()
{
// ...
$this->addCopyField(SiteTree::class . '_Title', 'spellcheckData');
$this->addCopyField(SomeModel::class . '_Title', 'spellcheckData');
$this->addCopyField(SiteTree::class . '_Content', 'spellcheckData');
$this->addCopyField(SomeModel::class . '_Content', 'spellcheckData');
}
// ...
// ...
public function getFieldDefinitions()
{
$xml = parent::getFieldDefinitions();
function getFieldDefinitions() {
$xml = parent::getFieldDefinitions();
$xml .= "\n\n\t\t<!-- Additional custom fields for spell checking -->";
$xml .= "\n\t\t<field name='spellcheckData' type='textSpellHtml' indexed='true' stored='false' multiValued='true' />";
$xml .= "\n\n\t\t<!-- Additional custom fields for spell checking -->";
$xml .= "\n\t\t<field name='spellcheckData' type='textSpellHtml' indexed='true' stored='false' multiValued='true' />";
return $xml;
}
}
return $xml;
}
}
```
Now you need to tell solr to use our new field for gathering spelling data.
In order to customize the spell checking configuration,
create your own `solrconfig.xml` (see "File-based configuration").
In there, change the following directive:
<!-- ... -->
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<!-- ... -->
<str name="field">spellcheckData</str>
</searchComponent
```xml
<!-- ... -->
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<!-- ... -->
<str name="field">spellcheckData</str>
</searchComponent>
```
Don't forget to copy the new configuration via a call to the `Solr_Configure`
task, and reindex your data before using the spell checker.
task, and reindex your data before using the spell checker.
### Limiting search fields
@ -287,13 +332,20 @@ specified in the form of `{table}_{field}`.
These fields are defined in the schema.xml file that gets sent to Solr.
$query = new SearchQuery();
$query->classes = array(array('class' => 'Page', 'includeSubclasses' => true));
$query->search('someterms', array('SiteTree_Title', 'SiteTree_Content'));
$result = singleton('SolrSearchIndex')->search($query, -1, -1);
```php
use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
// the request to Solr would be:
// q=(SiteTree_Title:Lorem+OR+SiteTree_Content:Lorem)
$query = new SearchQuery();
$query->classes = [
['class' => Page::class, 'includeSubclasses' => true],
];
$query->search('someterms', [SiteTree::class . '_Title', SiteTree::class . '_Content']);
$result = singleton(SolrSearchIndex::class)->search($query, -1, -1);
// the request to Solr would be:
// q=(SiteTree_Title:Lorem+OR+SiteTree_Content:Lorem)
```
### Configuring boosts
@ -309,13 +361,20 @@ to the top of the results.
In this example, we enter "Lorem" as the search term, and boost the `Content` field:
$query = new SearchQuery();
$query->classes = array(array('class' => 'Page', 'includeSubclasses' => true));
$query->search('Lorem', null, array('SiteTree_Content' => 2));
$result = singleton('SolrSearchIndex')->search($query, -1, -1);
```php
use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
// the request to Solr would be:
// q=SiteTree_Content:Lorem^2
$query = new SearchQuery();
$query->classes = [
['class' => 'Page', 'includeSubclasses' => true],
];
$query->search('Lorem', null, [SiteTree::class . '_Content' => 2]);
$result = singleton(SolrSearchIndex::class)->search($query, -1, -1);
// the request to Solr would be:
// q=SiteTree_Content:Lorem^2
```
More information on [relevancy on the Solr wiki](http://wiki.apache.org/solr/SolrRelevancyFAQ).
@ -333,20 +392,23 @@ with the key `boost` assigned to the desired value.
For example:
```php
use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
:::php
class SolrSearchIndex extends SolrIndex {
public function init() {
$this->addClass('SiteTree');
$this->addAllFulltextFields();
$this->addFilterField('ShowInSearch');
this->addBoostedField('Title', null, array(), 1.5);
this->setFieldBoosting('SiteTree_SearchBoost', 2);
}
}
class SolrSearchIndex extends SolrIndex
{
public function init()
{
$this->addClass(SiteTree::class);
$this->addAllFulltextFields();
$this->addFilterField('ShowInSearch');
$this->addBoostedField('Title', null, [], 1.5);
$this->setFieldBoosting(SiteTree::class . '_SearchBoost', 2);
}
}
```
### Custom Types
@ -358,12 +420,18 @@ by overloading the template responsible for it: `types.ss`.
In the following example, we read out type definitions
from a new file `mysite/solr/templates/types.ss` instead:
<?php
class MyIndex extends SolrIndex {
function getTemplatesPath() {
return Director::baseFolder() . '/mysite/solr/templates/';
}
}
```php
use SilverStripe\Control\Director;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function getTemplatesPath()
{
return Director::baseFolder() . '/mysite/solr/templates/';
}
}
```
### Highlighting
@ -373,22 +441,31 @@ the term is used). In order to use this feature, the full content of the
field to be highlighted needs to be stored in the index,
by declaring it through `addStoredField()`.
<?php
class MyIndex extends SolrIndex {
function init() {
$this->addClass('Page');
$this->addAllFulltextFields();
$this->addStoredField('Content');
}
}
```php
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addAllFulltextFields();
$this->addStoredField('Content');
}
}
```
To search with highlighting enabled, you need to pass in a custom query parameter.
There's a lot more parameters to tweak results on the [Solr Wiki](http://wiki.apache.org/solr/HighlightingParameters).
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$results = $index->search($query, -1, -1, array('hl' => 'true'));
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$results = $index->search($query, -1, -1, ['hl' => 'true']);
```
Each result will automatically contain an "Excerpt" property
which you can use in your own results template.
@ -404,72 +481,91 @@ is used to send data to Solr and parse the response. Additional information can
be pulled from this response and added to your results object for use in templates
using the `updateSearchResults()` extension hook.
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$results = $index->search($query, -1, -1, array(
'facet' => 'true',
'facet.field' => 'SiteTree_ClassName',
));
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$index = new MyIndex();
$query = new SearchQuery();
$query->search('My Term');
$results = $index->search($query, -1, -1, [
'facet' => 'true',
'facet.field' => 'SiteTree_ClassName',
]);
```
By adding facet fields into the query parameters, our response object from Solr
now contains some additional information that we can add into the results sent
to the page.
<?php
class MyResultsExtension extends Extension {
/**
* Adds extra information from the solr-php-client repsonse
* into our search results.
* @param $results The ArrayData that will be used to generate search
* results pages.
* @param $response The solr-php-client response object.
*/
public function updateSearchResults($results, $response)
{
if (!isset($response->facet_counts) || !isset($response->facet_counts->facet_fields)) {
return;
}
$facetCounts = ArrayList::create(array());
foreach($response->facet_counts->facet_fields as $name => $facets) {
$facetDetails = ArrayData::create(array(
'Name' => $name,
'Facets' => ArrayList::create(array()),
));
foreach($facets as $facetName => $facetCount) {
$facetDetails->Facets->push(ArrayData::create(array(
'Name' => $facetName,
'Count' => $facetCount,
)));
}
$facetCounts->push($facetDetails);
}
$results->setField('FacetCounts', $facetCounts);
}
}
```php
use SilverStripe\Core\Extension;
use SilverStripe\View\ArrayData;
use SilverStripe\ORM\ArrayList;
class MyResultsExtension extends Extension
{
/**
* Adds extra information from the solr-php-client repsonse
* into our search results.
* @param ArrayData $results The ArrayData that will be used to generate search
* results pages.
* @param stdClass $response The solr-php-client response object.
*/
public function updateSearchResults($results, $response)
{
if (!isset($response->facet_counts) || !isset($response->facet_counts->facet_fields)) {
return;
}
$facetCounts = ArrayList::create(array());
foreach($response->facet_counts->facet_fields as $name => $facets) {
$facetDetails = ArrayData::create([
'Name' => $name,
'Facets' => ArrayList::create([]),
]);
foreach($facets as $facetName => $facetCount) {
$facetDetails->Facets->push(ArrayData::create([
'Name' => $facetName,
'Count' => $facetCount,
]));
}
$facetCounts->push($facetDetails);
}
$results->setField('FacetCounts', $facetCounts);
}
}
```
We can now access the facet information inside our templates.
### Adding Analyzers, Tokenizers and Token Filters
When a document is indexed, its individual fields are subject to the analyzing and tokenizing filters that can transform and normalize the data in the fields. For example — removing blank spaces, removing html code, stemming, removing a particular character and replacing it with another
When a document is indexed, its individual fields are subject to the analyzing and tokenizing filters that can transform and normalize the data in the fields. For example — removing blank spaces, removing html code, stemming, removing a particular character and replacing it with another
(see [Solr Wiki](http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters)).
Example: Replace synonyms on indexing (e.g. "i-pad" with "iPad")
<?php
class MyIndex extends SolrIndex {
function init() {
$this->addClass('Page');
$this->addField('Content');
$this->addAnalyzer('Content', 'filter', array('class' => 'solr.SynonymFilterFactory'));
}
}
```php
use SilverStripe\FullTextSearch\Solr\SolrIndex;
// Generates the following XML schema definition:
// <field name="Page_Content" ...>
// <filter class="solr.SynonymFilterFactory" synonyms="syn.txt" ignoreCase="true" expand="false"/>
// </field>
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addField('Content');
$this->addAnalyzer('Content', 'filter', ['class' => 'solr.SynonymFilterFactory']);
}
}
```
Generates the following XML schema definition:
```xml
<field name="Page_Content" ...>
<filter class="solr.SynonymFilterFactory" synonyms="syn.txt" ignoreCase="true" expand="false"/>
</field>
```
### Text Extraction
@ -480,24 +576,30 @@ If you're using a default Solr installation, it's most likely already
bundled and set up. But if you plan on running the Solr server integrated
into this module, you'll need to download the libraries and link the first.
wget http://archive.apache.org/dist/lucene/solr/3.1.0/apache-solr-3.1.0.tgz
mkdir tmp
tar -xvzf apache-solr-3.1.0.tgz
mkdir .solr/PageSolrIndexboot/dist
mkdir .solr/PageSolrIndexboot/contrib
cp apache-solr-3.1.0/dist/apache-solr-cell-3.1.0.jar .solr/PageSolrIndexboot/dist/
cp -R apache-solr-3.1.0/contrib/extraction .solr/PageSolrIndexboot/contrib/
rm -rf apache-solr-3.1.0 apache-solr-3.1.0.tgz
```
wget http://archive.apache.org/dist/lucene/solr/3.1.0/apache-solr-3.1.0.tgz
mkdir tmp
tar -xvzf apache-solr-3.1.0.tgz
mkdir .solr/PageSolrIndexboot/dist
mkdir .solr/PageSolrIndexboot/contrib
cp apache-solr-3.1.0/dist/apache-solr-cell-3.1.0.jar .solr/PageSolrIndexboot/dist/
cp -R apache-solr-3.1.0/contrib/extraction .solr/PageSolrIndexboot/contrib/
rm -rf apache-solr-3.1.0 apache-solr-3.1.0.tgz
```
Create a custom `solrconfig.xml` (see "File-based configuration").
Add the following XML configuration.
<lib dir="./contrib/extraction/lib/" />
<lib dir="./dist" />
```xml
<lib dir="./contrib/extraction/lib/" />
<lib dir="./dist" />
```
Now apply the configuration:
sake dev/tasks/Solr_Configure
```
vendor/bin/sake dev/tasks/Solr_Configure
```
Now you can use Solr text extraction either directly through the HTTP API,
or indirectly through the ["textextraction" module](https://github.com/silverstripe-labs/silverstripe-textextraction).
@ -509,23 +611,29 @@ index. You'll have to make some changes to add it in.
So, let's take an example of `StaffMember`:
:::php
<?php
class StaffMember extends DataObject {
private static $db = array(
'Name' => 'Varchar(255)',
'Abstract' => 'Text',
'PhoneNumber' => 'Varchar(50)'
);
public function Link($action = 'show') {
return Controller::join_links('my-controller', $action, $this->ID);
}
public function getShowInSearch() {
return 1;
}
}
```php
use SilverStripe\Control\Controller;
use SilverStripe\ORM\DataObject;
class StaffMember extends DataObject
{
private static $db = [
'Name' => 'Varchar(255)',
'Abstract' => 'Text',
'PhoneNumber' => 'Varchar(50)',
];
public function Link($action = 'show')
{
return Controller::join_links('my-controller', $action, $this->ID);
}
public function getShowInSearch()
{
return 1;
}
}
```
This `DataObject` class has the minimum code necessary to allow it to be viewed in the site search.
@ -536,25 +644,29 @@ search result title.
So with that, let's create a new class called `MySolrSearchIndex`:
:::php
<?php
class MySolrSearchIndex extends SolrIndex {
public function init() {
$this->addClass('SiteTree');
$this->addClass('StaffMember');
$this->addAllFulltextFields();
$this->addFilterField('ShowInSearch');
}
}
```php
use StaffMember;
use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MySolrSearchIndex extends SolrIndex {
public function init()
{
$this->addClass(SiteTree::class);
$this->addClass(StaffMember::class);
$this->addAllFulltextFields();
$this->addFilterField('ShowInSearch');
}
}
```
This is a copy/paste of the existing configuration but with the addition of `StaffMember`.
Once you've created the above classes and run `flush=1`, access `dev/tasks/Solr_Configure` and `dev/tasks/Solr_Reindex`
to tell Solr about the new index you've just created. This will add `StaffMember` and the text fields it has to the
index. Now when you search on the site using `MySolrSearchIndex->search()`,
index. Now when you search on the site using `MySolrSearchIndex->search()`,
the `StaffMember` results will show alongside normal `Page` results.
@ -566,12 +678,14 @@ You can visit `http://localhost:8983/solr`, which will show you a list
to the admin interfaces of all available indices.
There you can search the contents of the index via the native SOLR web interface.
It is possible to manually replicate the data automatically sent
to Solr when saving/publishing in SilverStripe,
which is useful when debugging front-end queries,
It is possible to manually replicate the data automatically sent
to Solr when saving/publishing in SilverStripe,
which is useful when debugging front-end queries,
see `thirdparty/fulltextsearch/server/silverstripe-solr-test.xml`.
java -Durl=http://localhost:8983/solr/MyIndex/update/ -Dtype=text/xml -jar post.jar silverstripe-solr-test.xml
```
java -Durl=http://localhost:8983/solr/MyIndex/update/ -Dtype=text/xml -jar post.jar silverstripe-solr-test.xml
```
## FAQ
@ -582,7 +696,10 @@ so the field might not exist in all your index entries.
A simple bounded range query (`<field>:[* TO <date>]`) will fail in this case.
In order to query the field, reverse the search conditions and exclude the ranges you don't want:
// Wrong: Filter will ignore all empty field values
$myQuery->filter(<field>, new SearchQuery_Range('*', <date>));
// Better: Exclude the opposite range
$myQuery->exclude(<field>, new SearchQuery_Range(<date>, '*'));
```php
// Wrong: Filter will ignore all empty field values
$myQuery->filter('fieldname', new SearchQuery_Range('*', 'somedate'));
// Better: Exclude the opposite range
$myQuery->exclude('fieldname', new SearchQuery_Range('somedate', '*'));
```

View File

@ -19,10 +19,10 @@ design and the object model meant that searching was inefficient. The abstractio
hard to then figure out what was going on.
This module instead provides the ability to define those indexes and queries in PHP. The indexes are defined as a mapping
between the SilverStripe object model and the connector-specific fulltext engine index model. This module then interrogates model metadata
to build the specific index definition.
between the SilverStripe object model and the connector-specific fulltext engine index model. This module then interrogates model metadata
to build the specific index definition.
It also hooks into SilverStripe framework in order to update the indexes when the models change and connectors then convert those index and query definitions
It also hooks into SilverStripe framework in order to update the indexes when the models change and connectors then convert those index and query definitions
into fulltext engine specific code.
The intent of this module is not to make changing fulltext search engines seamless. Where possible this module provides
@ -36,34 +36,49 @@ Basic usage is a four step process:
1). Define an index in SilverStripe (Note: The specific connector index instance - that's what defines which engine gets used)
// File: mysite/code/MyIndex.php:
<?php
class MyIndex extends SolrIndex {
function init() {
$this->addClass('Page');
$this->addFulltextField('Title');
$this->addFulltextField('Content');
}
}
```php
// File: mysite/code/MyIndex.php:
use Page;
use SilverStripe\FullTextSearch\Solr\SolrIndex;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addFulltextField('Title');
$this->addFulltextField('Content');
}
}
```
You can also skip listing all searchable fields, and have the index
figure it out automatically via `addAllFulltextFields()`.
2). Add something to the index (Note: You can also just update an existing document in the CMS. but adding _existing_ objects to the index is connector specific)
$page = new Page(array('Content' => 'Help me. My house is on fire. This is less than optimal.'));
$page->write();
```php
$page = Page::create(['Content' => 'Help me. My house is on fire. This is less than optimal.']);
$page->write();
```
Note: There's usually a connector-specific "reindex" task for this.
3). Build a query
$query = new SearchQuery();
$query->search('My house is on fire');
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = new SearchQuery();
$query->search('My house is on fire');
```
4). Apply that query to an index
$results = singleton('MyIndex')->search($query);
```php
$results = singleton(MyIndex::class)->search($query);
```
Note that for most connectors, changes won't be searchable until _after_ the request that triggered the change.
@ -80,36 +95,49 @@ In order to render search results, you need to return them from a controller.
You can also drive this through a form response through standard SilverStripe forms.
In this case we simply assume there's a GET parameter named `q` with a search term present.
class Page_Controller extends ContentController {
private static $allowed_actions = array('search');
public function search($request) {
$query = new SearchQuery();
$query->search($request->getVar('q'));
return $this->renderWith('array(
'SearchResult' => singleton('MyIndex')->search($query);
);
}
}
```php
use SilverStripe\CMS\Controllers\ContentController;
use SilverStripe\Control\HTTPRequest;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
class PageController extends ContentController
{
private static $allowed_actions = [
'search',
];
public function search(HTTPRequest $request)
{
$query = new SearchQuery();
$query->search($request->getVar('q'));
return $this->renderWith([
'SearchResult' => singleton(MyIndex::class)->search($query)
]);
}
}
```
In your template (e.g. `Page_results.ss`) you can access the results and loop through them.
They're stored in the `$Matches` property of the search return object.
<% if SearchResult.Matches %>
<h2>Results for &quot;{$Query}&quot;</h2>
<p>Displaying Page $SearchResult.Matches.CurrentPage of $SearchResult.Matches.TotalPages</p>
<ol>
<% loop SearchResult.Matches %>
<li>
<h3><a href="$Link">$Title</a></h3>
<p><% if Abstract %>$Abstract.XML<% else %>$Content.ContextSummary<% end_if %></p>
</li>
<% end_loop %>
</ol>
<% else %>
<p>Sorry, your search query did not return any results.</p>
<% end_if %>
Please check the [pagination guide](http://docs.silverstripe.org/en/3.2/developer_guides/templates/how_tos/pagination/)
```ss
<% if $SearchResult.Matches %>
<h2>Results for &quot;{$Query}&quot;</h2>
<p>Displaying Page $SearchResult.Matches.CurrentPage of $SearchResult.Matches.TotalPages</p>
<ol>
<% loop $SearchResult.Matches %>
<li>
<h3><a href="$Link">$Title</a></h3>
<p><% if $Abstract %>$Abstract.XML<% else %>$Content.ContextSummary<% end_if %></p>
</li>
<% end_loop %>
</ol>
<% else %>
<p>Sorry, your search query did not return any results.</p>
<% end_if %>
```
Please check the [pagination guide](https://docs.silverstripe.org/en/4/developer_guides/templates/how_tos/pagination/)
in the main SilverStripe documentation to learn how to paginate through search results.
## Automatic Index Updates
@ -123,7 +151,7 @@ For example, a CMS author might have edited a page, or a user has left a new com
In order to minimise delays to those users, the index update is deferred until after
the actual request returns to the user, through PHP's `register_shutdown_function()` functionality.
If the [queuedjobs](https://github.com/silverstripe-australia/silverstripe-queuedjobs) module is installed,
If the [queuedjobs](https://github.com/symbiote/silverstripe-queuedjobs) module is installed,
updates are queued up instead of executed in the same request. Queue jobs are usually processed every minute.
Large index updates will be batched into multiple queue jobs to ensure a job can run to completion within
common execution constraints (memory and time limits). You can check the status of jobs in
@ -138,23 +166,32 @@ Manual updates are connector specific, please check the connector docs for detai
By default, the index searches through all indexed fields.
This can be limited by arguments to the `search()` call.
$query = new SearchQuery();
$query->search('My house is on fire', array('Page_Title'));
// No results, since we're searching in title rather than page content
$results = singleton('MyIndex')->search($query);
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = new SearchQuery();
$query->search('My house is on fire', [Page::class . '_Title']);
// No results, since we're searching in title rather than page content
$results = singleton(MyIndex::class)->search($query);
```
## Searching Value Ranges
Most values can be expressed as ranges, most commonly dates or numbers.
To search for a range of values rather than an exact match,
To search for a range of values rather than an exact match,
use the `SearchQuery_Range` class. The range can include bounds on both sides,
or stay open ended by simply leaving the argument blank.
$query = new SearchQuery();
$query->search('My house is on fire');
// Only include documents edited in 2011 or earlier
$query->filter('Page_LastEdited', new SearchQuery_Range(null, '2011-12-31T23:59:59Z'));
$results = singleton('MyIndex')->search($query);
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery_Range;
$query = new SearchQuery();
$query->search('My house is on fire');
// Only include documents edited in 2011 or earlier
$query->filter(Page::class . '_LastEdited', new SearchQuery_Range(null, '2011-12-31T23:59:59Z'));
$results = singleton(MyIndex::class)->search($query);
```
Note: At the moment, the date format is specific to the search implementation.
@ -165,53 +202,69 @@ and the search index persistence, its often not clear which condition is searche
Should it equal an empty string, or only match if the field wasn't indexed at all?
The `SearchQuery` API has the concept of a "missing" and "present" field value for this:
$query = new SearchQuery();
$query->search('My house is on fire');
// Needs a value, although it can be false
$query->filter('Page_ShowInMenus', SearchQuery::$present);
$results = singleton('MyIndex')->search($query);
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = new SearchQuery();
$query->search('My house is on fire');
// Needs a value, although it can be false
$query->filter(Page::class . '_ShowInMenus', SearchQuery::$present);
$results = singleton(MyIndex::class)->search($query);
```
## Indexing Multiple Classes
An index is a denormalized view of your data, so can hold data from more than one model.
As you can only search one index at a time, all searchable classes need to be included.
// File: mysite/code/MyIndex.php:
<?php
class MyIndex extends SolrIndex {
function init() {
$this->addClass('Page');
$this->addClass('Member');
$this->addFulltextField('Content'); // only applies to Page class
$this->addFulltextField('FirstName'); // only applies to Member class
}
}
```php
// File: mysite/code/MyIndex.php
use SilverStripe\FullTextSearch\Solr\SolrIndex;
use SilverStripe\Security\Member;
class MyIndex extends SolrIndex
{
public function init()
{
$this->addClass(Page::class);
$this->addClass(Member::class);
$this->addFulltextField('Content'); // only applies to Page class
$this->addFulltextField('FirstName'); // only applies to Member class
}
}
```
## Using Multiple Indexes
Multiple indexes can be created and searched independently, but if you wish to override an existing
index with another, you can use the `$hide_ancestor` config.
:::php
class MyReplacementIndex extends MyIndex {
private static $hide_ancestor = 'MyIndex';
```php
use SilverStripe\Assets\File;
public function init() {
parent::init();
$this->addClass('File');
$this->addFulltextField('Title');
}
}
class MyReplacementIndex extends MyIndex
{
private static $hide_ancestor = 'MyIndex';
You can also filter all indexes globally to a set of pre-defined classes if you wish to
public function init()
{
parent::init();
$this->addClass(File::class);
$this->addFulltextField('Title');
}
}
```
You can also filter all indexes globally to a set of pre-defined classes if you wish to
prevent any unknown indexes from being automatically included.
:::yaml
FullTextSearch:
indexes:
- MyReplacementIndex
- CoreSearchIndex
```yaml
SilverStripe\FullTextSearch\Search\FullTextSearch:
indexes:
- MyReplacementIndex
- CoreSearchIndex
```
## Indexing Relationships
@ -229,16 +282,20 @@ anthing above increases it.
Example:
$query = new SearchQuery();
$query->search(
'My house is on fire',
null,
array(
'Page_Title' => 1.5,
'Page_Content' => 1.0
)
);
$results = singleton('MyIndex')->search($query);
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
$query = new SearchQuery();
$query->search(
'My house is on fire',
null,
[
Page::class . '_Title' => 1.5,
Page::class . '_Content' => 1.0,
]
);
$results = singleton(MyIndex::class)->search($query);
```
## Filtering
@ -263,15 +320,27 @@ For most cases, you'll want to exclude draft content from your search results.
You can either prevent the draft content from being indexed in the first place,
by adding the following to your `SearchIndex->init()` method:
$this->excludeVariantState(array('SearchVariantVersioned' => 'Stage'));
```php
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
Alternatively, you can index draft content, but simply exclude it from searches.
$this->excludeVariantState([SearchVariantVersioned::class => 'Stage']);
```
Alternatively, you can index draft content, but simply exclude it from searches.
This can be handy to preview search results on unpublished content, in case a CMS author is logged in.
Before constructing your `SearchQuery`, conditionally switch to the "live" stage:
if(!Permission::check('CMS_ACCESS_CMSMain')) Versioned::reading_stage('Live');
$query = new SearchQuery();
// ...
```php
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
use SilverStripe\Security\Permission;
use SilverStripe\Versioned\Versioned;
if (!Permission::check('CMS_ACCESS_CMSMain')) {
Versioned::set_stage(Versioned::LIVE);
}
$query = new SearchQuery();
// ...
```
### How do I write nested/complex filters?