mirror of
https://github.com/silverstripe/silverstripe-fulltextsearch
synced 2024-10-22 14:05:29 +02:00
Merge pull request #281 from creative-commoners/pulls/3/indexable-canview
SearchableService canView()
This commit is contained in:
Maxime Rainville
GitHub
commit
f274f85977
76
README.md
76
README.md
@ -8,9 +8,81 @@
|
||||
Adds support for fulltext search engines like Sphinx and Solr to SilverStripe CMS.
|
||||
Compatible with PHP 7.2
|
||||
|
||||
## Maintainer Contact
|
||||
## Important notes when upgrading to fulltextsearch 3.7.0+
|
||||
|
||||
* Hamish Friedlander <hamish (at) silverstripe (dot) com>
|
||||
There are some significant changes from previous versions:
|
||||
|
||||
Draft content will no longer be automatically added to the search index. This new behaviour was previously an
|
||||
opt-in behaviour that was enabled by adding the following line to a search index:
|
||||
|
||||
```
|
||||
$this->excludeVariantState([SearchVariantVersioned::class => Versioned::DRAFT]);
|
||||
```
|
||||
|
||||
A new `canView()` check against an anonymous user (i.e. someone not logged in) and a `ShowInSearch` check is now
|
||||
performed by default against all records (DataObjects) before being added to the search index, and also before being
|
||||
shown in search results. This may mean that some records that were previously being indexed and shown in search results
|
||||
will no longer appear due to these additional checks.
|
||||
|
||||
These additional checks have been added with data security in mind, and it's assumed that records failing these
|
||||
checks probably should not be indexed in the first place.
|
||||
|
||||
# Enable indexing of draft content:
|
||||
|
||||
You can index draft content with the following yml configuration:
|
||||
|
||||
```
|
||||
SilverStripe\FullTextSearch\Search\Services\SearchableService:
|
||||
variant_state_draft_excluded: false
|
||||
```
|
||||
|
||||
However, when set to false, it will still only index draft content when a DataObject is in a published state, not a
|
||||
draft-only or modified state. This is because it will still fail the new anonymous user `canView()` check in
|
||||
`SearchableService::isSearchable()` and be automatically deleted from the index.
|
||||
|
||||
If you wish to also index draft content when a DataObject is in a draft-only or a modified state, then you'll need
|
||||
to also configure `SearchableService::indexing_canview_exclude_classes`. See below for instructions on how to do this.
|
||||
|
||||
# Disabling the anonymous user canView() pre-index check
|
||||
|
||||
You can apply configuration to remove the new pre-index `canView()` check from your DataObjects if it is not necessary,
|
||||
or if it impedes expected functionality (e.g. for sites where users must authenticate to view any content). This will
|
||||
also disable the check for descendants of the specified DataObjects. Ensure that your implementation of fulltextsearch
|
||||
is correctly performing a `canView()` check at query time before disabling the pre-index check, as this may result in
|
||||
leakage of private data.
|
||||
|
||||
```
|
||||
SilverStripe\FullTextSearch\Search\Services\SearchableService:
|
||||
indexing_canview_exclude_classes:
|
||||
- Some\Org\MyDataObject
|
||||
# This will disable the check for all pagetypes:
|
||||
- SilverStripe\CMS\Model\SiteTree
|
||||
```
|
||||
|
||||
You can also use the `updateIsSearchable` extension point on `SearchableService` to modify the result of the method
|
||||
after the `ShowInSearch` and `canView()` checks have run.
|
||||
|
||||
It is highly recommend you run a [solr_reindex](https://github.com/silverstripe/silverstripe-fulltextsearch/blob/3/docs/en/03_configuration.md#solr-reindex)
|
||||
on your production site after upgrading from 3.6 or earlier to purge any old data that should no longer be in the search index.
|
||||
|
||||
These additional check can have an impact on the reindex performance due to additional queries for permission checks.
|
||||
If your site also indexes content in files, such as pdf's or docx's, using the [text-extraction](https://github.com/silverstripe/silverstripe-textextraction)
|
||||
module which is fairly time-intensive, then the relative performance impact of the `canView()` checks won't be as noticeable.
|
||||
|
||||
## Details on filtering before adding content to the solr index
|
||||
- `SearchableService::isIndexable()` check in `SolrReindexBase`. Used when indexing all records during Solr reindex.
|
||||
- `SearchableService::isIndexable()` check in `SearchUpdateProcessor`. Used when indexing single records during
|
||||
`DataObject->write()`.
|
||||
|
||||
## Details on filtering when extracting results from the solr index
|
||||
- `SearchableService::isViewable()` check in `SolrIndex`. This will often be used in CWP implementations that use the
|
||||
`CwpSearchEngine` class, as well as most custom implementations that call `MySearchIndex->search()`
|
||||
- `SearchableService::isViewable()` check in `SearchForm`. This will be used in solr implementations where a
|
||||
`/SearchForm` url is used to display search results.
|
||||
- Some implementations will call `SearchableService::isViewable()` twice. If this happens then the first call will be
|
||||
cached in memory so there is virtually no performance penalty calling it a second time.
|
||||
- If your implementation is very custom and does not subclass nor make use of either `SolrIndex` or `SearchForm`, then
|
||||
it's recommended you update your implementation to call `SearchableService::isViewable()`.
|
||||
|
||||
## Requirements
|
||||
|
||||
|
||||
90
changelog.md
90
changelog.md
@ -1,90 +0,0 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
This project adheres to [Semantic Versioning](http://semver.org/).
|
||||
|
||||
## [2.2.0]
|
||||
|
||||
* FIX Indexes with custom index names that don't match the classname were breaking
|
||||
* BUG Fix versioned writes where subtables have no fields key
|
||||
* BUGFIX: Fixed issue where the $id variable would be overridden in sub sequent iterations of the derived fields loop
|
||||
* adding stemming support
|
||||
* BUG fix issues with search variants applying to more than one class
|
||||
* API adding stemming support
|
||||
* FIX: Fix initial dev/build on PDO Database.
|
||||
|
||||
## [2.1.1]
|
||||
|
||||
* Converted to PSR-2
|
||||
* FIX: remove parameters from function calls
|
||||
* Added standard code of conduct
|
||||
* Added standard editor config
|
||||
* Updated license
|
||||
* Added standard gitattributes
|
||||
* MINOR: Don't include Hamcrest globally so it doesn't conflict with PHPUnit
|
||||
|
||||
## [2.1.0]
|
||||
|
||||
* 3.2 Compatibility
|
||||
* Add ss 3.2 and PHP 5.6 to CI
|
||||
* Added standard Scrutinizer config
|
||||
* Added standard Travis config
|
||||
|
||||
## [2.0.0]
|
||||
|
||||
* Fix highlight support when querying by fields (or boosting fields)
|
||||
* Updating travis provisioner
|
||||
* Added docs about controller and template usage
|
||||
* API Enable boosted fields to be specified on the index
|
||||
* BUG Prevent subsites breaking solrindexversionedtest
|
||||
* Enable indexes to upload custom config
|
||||
* API Additional support for custom copy_fields
|
||||
* API QueuedJob support for Solr_Reindex
|
||||
|
||||
## [1.1.0]
|
||||
|
||||
* API Solr_Reindex uses configured SearchUpdater instead of always doing a direct write
|
||||
* Fix class limit on delete query in SolrIndex
|
||||
* Regression in SearchUpdater_ObjectHandler
|
||||
* API Separate searchupdate / commit into separate queued-jobs
|
||||
* API Only allow one scheduled commit job at a time
|
||||
|
||||
## [1.0.6]
|
||||
|
||||
* Make spelling suggestions more useful
|
||||
* BUG Add missing addStoredFields method
|
||||
|
||||
## [1.0.5]
|
||||
|
||||
* BUG Fix Solr 4.0 compatibility issue
|
||||
* BUG Fix test case not elegantly failing on missing phockito
|
||||
* API SearchUpdateQueuedJobProcessor now uses batching
|
||||
* Fix many_many fieldData bug
|
||||
* Adding tests for SearchIndex::fieldData()
|
||||
* Add a no-op query to prevent database timeouts during a long reindex
|
||||
|
||||
## [1.0.4]
|
||||
|
||||
* BUG Patch up the information leak of debug information.
|
||||
* FIX: will work for postgreSQL
|
||||
|
||||
## [1.0.3]
|
||||
|
||||
Users upgrading from 1.0.2 or below will need to run the Solr_Reindex task to refresh
|
||||
each SolrIndex. This is due to a change in record IDs, which are now generated from
|
||||
the base class of each DataObject, rather than the instance class, as well as fixes
|
||||
to integration with the subsites module.
|
||||
|
||||
Developers working locally should be aware that by default, all indexes will be updated
|
||||
in realtime when the environment is in dev mode, rather than attempting to queue these
|
||||
updates with the queued jobs module (if installed).
|
||||
|
||||
### Bugfixes
|
||||
|
||||
* BUG Fix old indexing storing against the incorrect class key
|
||||
* [Don't rely on MySQL ordering of index->getAdded()](https://github.com/silverstripe-labs/silverstripe-fulltextsearch/commit/4b51393e014fc4c0cc8e192c74eb4594acaca605)
|
||||
|
||||
### API
|
||||
|
||||
* [API Disable queued processing for development environments](https://github.com/silverstripe-labs/silverstripe-fulltextsearch/commit/71fc359b3711cf5b9429d86da0f1e0b20bd43dee)
|
||||
@ -22,7 +22,7 @@
|
||||
],
|
||||
"require": {
|
||||
"php": ">=7.1",
|
||||
"silverstripe/framework": "^4.0",
|
||||
"silverstripe/framework": "^4",
|
||||
"monolog/monolog": "~1.15",
|
||||
"ptcinc/solr-php-client": "^1.0",
|
||||
"symfony/process": "^3.2",
|
||||
|
||||
@ -112,12 +112,13 @@ Depending on the size of the index and how much content needs to be processed, i
|
||||
|
||||
If the [Queued Jobs module](https://github.com/symbiote/silverstripe-queuedjobs/) is installed, updates are queued up instead of executed in the same request. Queued jobs are usually processed every minute. Large index updates will be batched into multiple queued jobs to ensure a job can run to completion within common constraints, such as memory and execution time limits. You can check the status of jobs in an administrative interface under `admin/queuedjobs/`.
|
||||
|
||||
### Excluding draft content
|
||||
### Draft content
|
||||
|
||||
By default, the `SearchUpdater` class indexes all available "variant states", so in the case of the `Versioned` extension, both "draft" and "live".
|
||||
For most cases, you'll want to exclude draft content from your search results.
|
||||
By default, the `SearchUpdater` class attempts to index all available "variant states", except for draft content.
|
||||
Draft content is excluded by default via calls to SearchableService::variantStateExcluded().
|
||||
|
||||
You can either prevent the draft content from being indexed in the first place, by adding the following to your `SearchIndex::init()` method:
|
||||
Excluding draft content was a new default added in 3.7.0. Prior to that, draft content was previously indexed by
|
||||
default and could be excluded fron the index by adding the following to the `SearchIndex::init()` method:
|
||||
|
||||
```php
|
||||
use Page;
|
||||
@ -136,7 +137,10 @@ class MyIndex extends SolrIndex
|
||||
}
|
||||
```
|
||||
|
||||
Alternatively, you can index draft content, but simply exclude it from searches. This can be handy to preview search results on unpublished content, in case a CMS author is logged in. Before constructing your `SearchQuery`, conditionally switch to the "live" stage.
|
||||
If required, you can opt-out of the secure default and index draft content, but simply exclude it from searches.
|
||||
Read the inline documentation within SearchableService.php for more details on how to do this.
|
||||
This can be handy to preview search results on unpublished content, in case a CMS author is logged in.
|
||||
Before constructing your `SearchQuery`, conditionally switch to the "live" stage.
|
||||
|
||||
### Adding DataObjects
|
||||
|
||||
|
||||
@ -2,10 +2,12 @@
|
||||
|
||||
namespace SilverStripe\FullTextSearch\Search\Processors;
|
||||
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariant;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\Versioned\Versioned;
|
||||
|
||||
abstract class SearchUpdateProcessor
|
||||
{
|
||||
@ -72,7 +74,8 @@ abstract class SearchUpdateProcessor
|
||||
$dirtyIndexes = array();
|
||||
$dirty = $this->getSource();
|
||||
$indexes = FullTextSearch::get_indexes();
|
||||
$indexableService = IndexableService::singleton();
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
foreach ($dirty as $base => $statefulids) {
|
||||
if (!$statefulids) {
|
||||
continue;
|
||||
@ -86,11 +89,15 @@ abstract class SearchUpdateProcessor
|
||||
|
||||
// Ensure that indexes for all new / updated objects are included
|
||||
$objs = DataObject::get($base)->byIDs(array_keys($ids));
|
||||
|
||||
/** @var DataObject $obj */
|
||||
foreach ($objs as $obj) {
|
||||
foreach ($ids[$obj->ID] as $index) {
|
||||
if (!$indexes[$index]->variantStateExcluded($state)) {
|
||||
// Remove any existing records from index if ShowInSearch is changed to false
|
||||
if (!$indexableService->isIndexable($obj)) {
|
||||
if (!$searchableService->variantStateExcluded($state) &&
|
||||
!$indexes[$index]->variantStateExcluded($state)
|
||||
) {
|
||||
// Remove any existing data from index if the object is no longer indexable
|
||||
if (!$searchableService->isIndexable($obj)) {
|
||||
$indexes[$index]->delete($base, $obj->ID, $state);
|
||||
} else {
|
||||
$indexes[$index]->add($obj);
|
||||
@ -104,7 +111,9 @@ abstract class SearchUpdateProcessor
|
||||
// Generate list of records that do not exist and should be removed
|
||||
foreach ($ids as $id => $fromindexes) {
|
||||
foreach ($fromindexes as $index) {
|
||||
if (!$indexes[$index]->variantStateExcluded($state)) {
|
||||
if (!$searchableService->variantStateExcluded($state) &&
|
||||
!$indexes[$index]->variantStateExcluded($state)
|
||||
) {
|
||||
$indexes[$index]->delete($base, $id, $state);
|
||||
$dirtyIndexes[$index] = $indexes[$index];
|
||||
}
|
||||
|
||||
@ -1,58 +0,0 @@
|
||||
<?php
|
||||
|
||||
namespace SilverStripe\FullTextSearch\Search\Services;
|
||||
|
||||
use SilverStripe\Core\Extensible;
|
||||
use SilverStripe\Core\Injector\Injectable;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\ORM\Tests\MySQLDatabaseTest\Data;
|
||||
|
||||
/**
|
||||
* Checks if a DataObject is publically viewable thus able to be added or retrieved from a publically searchable index
|
||||
* Caching results because these checks may be done multiple times as there a few different code paths that search
|
||||
* results might follow in real-world search implementations
|
||||
*/
|
||||
class IndexableService
|
||||
{
|
||||
|
||||
use Injectable;
|
||||
use Extensible;
|
||||
|
||||
protected $cache = [];
|
||||
|
||||
public function clearCache(): void
|
||||
{
|
||||
$this->cache = [];
|
||||
}
|
||||
|
||||
public function isIndexable(DataObject $obj): bool
|
||||
{
|
||||
// check if is a valid DataObject that has been persisted to the database
|
||||
if (is_null($obj) || !$obj->ID) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$key = $this->getCacheKey($obj);
|
||||
if (isset($this->cache[$key])) {
|
||||
return $this->cache[$key];
|
||||
}
|
||||
|
||||
$value = true;
|
||||
|
||||
// This will also call $obj->getShowInSearch() if it exists
|
||||
if (isset($obj->ShowInSearch) && !$obj->ShowInSearch) {
|
||||
$value = false;
|
||||
}
|
||||
|
||||
$this->extend('updateIsIndexable', $obj, $value);
|
||||
$this->cache[$key] = $value;
|
||||
return $value;
|
||||
}
|
||||
|
||||
protected function getCacheKey(DataObject $obj): string
|
||||
{
|
||||
$key = $obj->ClassName . '_' . $obj->ID;
|
||||
$this->extend('updateCacheKey', $obj, $key);
|
||||
return $key;
|
||||
}
|
||||
}
|
||||
212
src/Search/Services/SearchableService.php
Normal file
212
src/Search/Services/SearchableService.php
Normal file
@ -0,0 +1,212 @@
|
||||
<?php
|
||||
|
||||
namespace SilverStripe\FullTextSearch\Search\Services;
|
||||
|
||||
use SilverStripe\Core\Config\Configurable;
|
||||
use SilverStripe\Core\Extensible;
|
||||
use SilverStripe\Core\Injector\Injectable;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\Security\Member;
|
||||
use SilverStripe\Versioned\Versioned;
|
||||
|
||||
/**
|
||||
* Checks if a DataObject is publicly viewable, thus able to be added to or retrieved from a publicly searchable index.
|
||||
* Results are cached because these checks may be run multiple times, as there a few different code paths that search
|
||||
* results might follow in real-world search implementations.
|
||||
*/
|
||||
class SearchableService
|
||||
{
|
||||
|
||||
use Injectable;
|
||||
use Extensible;
|
||||
use Configurable;
|
||||
|
||||
/**
|
||||
* Skip the canView() check at a class level to increase performance of search reindex.
|
||||
* Be careful as this may lead to content showing in search results that should not be there such as non-public,
|
||||
* cms-user-only content. This may potentially happen via edge cases such as skipping checks where subclasses
|
||||
* are involved.
|
||||
*
|
||||
* This has no effect on when search results as canView() must still be run there
|
||||
*
|
||||
* @var array namespaced classes to skip canView() check on search reindex
|
||||
*/
|
||||
private static $indexing_canview_exclude_classes = [];
|
||||
|
||||
/**
|
||||
* Configurable value to index draft content. Default is true for better security.
|
||||
*
|
||||
* If you need to index draft content, then view README.md for instructions
|
||||
*
|
||||
* @var bool
|
||||
*/
|
||||
private static $variant_state_draft_excluded = true;
|
||||
|
||||
/**
|
||||
* Non-persistant memory cache that only lasts the lifetime of the request
|
||||
*
|
||||
* @var array
|
||||
*/
|
||||
private $cache = [];
|
||||
|
||||
/**
|
||||
* Clears the internal cache
|
||||
*/
|
||||
public function clearCache(): void
|
||||
{
|
||||
$this->cache = [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Check to exclude a variant state
|
||||
*
|
||||
* @param array $state
|
||||
* @return bool
|
||||
*/
|
||||
public function variantStateExcluded(array $state): bool
|
||||
{
|
||||
if (self::config()->get('variant_state_draft_excluded') && $this->isDraftVariantState($state)) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a state array represents a draft variant
|
||||
*
|
||||
* @param array $state
|
||||
* @return bool
|
||||
*/
|
||||
private function isDraftVariantState(array $state): bool
|
||||
{
|
||||
$class = SearchVariantVersioned::class;
|
||||
return isset($state[$class]) && $state[$class] == Versioned::DRAFT;
|
||||
}
|
||||
|
||||
/**
|
||||
* Used during search reindex
|
||||
*
|
||||
* This is considered the primary layer of protection
|
||||
*
|
||||
* @param DataObject $obj
|
||||
* @return bool
|
||||
*/
|
||||
public function isIndexable(DataObject $obj): bool
|
||||
{
|
||||
return $this->isSearchable($obj, true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Used when retrieving search results
|
||||
*
|
||||
* This is considered the secondary layer of protection
|
||||
*
|
||||
* It's important to still have this layer in conjuction with the index layer as non-searchable results may be
|
||||
* in the search index because:
|
||||
* a) they were added to the index pre-fulltextsearch 3.7 and a reindex to purge old records was never run, OR
|
||||
* b) the DataObject has a non-deterministic canView() check such as `return $date <= $dateOfIndex;`
|
||||
*
|
||||
* @param DataObject $obj
|
||||
* @return bool
|
||||
*/
|
||||
public function isViewable(DataObject $obj): bool
|
||||
{
|
||||
return $this->isSearchable($obj, false);
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks and caches whether the given DataObject can be indexed. This is determined by two factors:
|
||||
* - Whether the ShowInSearch property / getShowInSearch() method evaluates to true
|
||||
* - Whether the canView method evaluates to true against an anonymous user (optional, can be disabled)
|
||||
*
|
||||
* @param DataObject $obj
|
||||
* @param bool $indexing
|
||||
* @return bool
|
||||
*/
|
||||
private function isSearchable(DataObject $obj, bool $indexing): bool
|
||||
{
|
||||
// check if is a valid DataObject that has been persisted to the database
|
||||
if (is_null($obj) || !$obj->ID) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$key = $this->getCacheKey($obj, $indexing);
|
||||
if (isset($this->cache[$key])) {
|
||||
return $this->cache[$key];
|
||||
}
|
||||
|
||||
$value = true;
|
||||
|
||||
// ShowInSearch check
|
||||
// This will also call $obj->getShowInSearch() if it exists
|
||||
if (isset($obj->ShowInSearch) && !$obj->ShowInSearch) {
|
||||
$value = false;
|
||||
}
|
||||
|
||||
// canView() checker
|
||||
if ($value) {
|
||||
$objClass = $obj->getClassName();
|
||||
if ($indexing) {
|
||||
// Anonymous member canView() for indexing
|
||||
if (!$this->classSkipsCanViewCheck($objClass)) {
|
||||
$value = Member::actAs(null, function () use ($obj) {
|
||||
return $obj->canView();
|
||||
});
|
||||
}
|
||||
} else {
|
||||
// Current member canView() check for retrieving search results
|
||||
$value = $obj->canView();
|
||||
}
|
||||
}
|
||||
$this->extend('updateIsSearchable', $obj, $indexing, $value);
|
||||
$this->cache[$key] = $value;
|
||||
return $value;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param DataObject $obj
|
||||
* @param bool $indexing
|
||||
* @return string
|
||||
*/
|
||||
private function getCacheKey(DataObject $obj, bool $indexing): string
|
||||
{
|
||||
$type = $indexing ? 'indexing' : 'viewing';
|
||||
// getUniqueKey() requires silverstripe/framework 4.6
|
||||
$uniqueKey = '';
|
||||
if (method_exists($obj, 'getUniqueKey')) {
|
||||
try {
|
||||
$uniqueKey = $obj->getUniqueKey();
|
||||
} catch (\Exception $e) {
|
||||
$uniqueKey = '';
|
||||
}
|
||||
}
|
||||
if (!$uniqueKey) {
|
||||
$uniqueKey = sprintf('%s-%s', $obj->ClassName, $obj->ID);
|
||||
}
|
||||
$key = sprintf('%s-%s', $type, $uniqueKey);
|
||||
$this->extend('updateCacheKey', $obj, $indexing, $key);
|
||||
return $key;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param string $class
|
||||
* @return bool
|
||||
*/
|
||||
private function classSkipsCanViewCheck(string $class): bool
|
||||
{
|
||||
$skipClasses = self::config()->get('indexing_canview_exclude_classes') ?? [];
|
||||
if (empty($skipClasses)) {
|
||||
return false;
|
||||
}
|
||||
if (in_array($class, $skipClasses)) {
|
||||
return true;
|
||||
}
|
||||
foreach ($skipClasses as $skipClass) {
|
||||
if (in_array($skipClass, class_parents($class))) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@ -9,7 +9,7 @@ use SilverStripe\Forms\FormAction;
|
||||
use SilverStripe\Forms\TextField;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Solr\SolrIndex;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\View\ArrayData;
|
||||
@ -83,13 +83,13 @@ class SearchForm extends Form
|
||||
$index = $indexClass::singleton();
|
||||
$results = $index->search($query, -1, -1, $params);
|
||||
|
||||
$indexableService = IndexableService::singleton();
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
// filter by permission
|
||||
if ($results) {
|
||||
foreach ($results->Matches as $match) {
|
||||
/** @var DataObject $match */
|
||||
if (!$indexableService->isIndexable($match)) {
|
||||
if (!$searchableService->isViewable($match)) {
|
||||
$results->Matches->remove($match);
|
||||
}
|
||||
}
|
||||
|
||||
@ -4,7 +4,8 @@ namespace SilverStripe\FullTextSearch\Solr\Reindex\Handlers;
|
||||
|
||||
use Psr\Log\LoggerInterface;
|
||||
use SilverStripe\Core\Environment;
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
use SilverStripe\FullTextSearch\Solr\Solr;
|
||||
use SilverStripe\FullTextSearch\Solr\SolrIndex;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariant;
|
||||
@ -12,6 +13,7 @@ use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\ORM\DataList;
|
||||
use SilverStripe\ORM\DB;
|
||||
use SilverStripe\Versioned\Versioned;
|
||||
|
||||
/**
|
||||
* Base class for re-indexing of solr content
|
||||
@ -41,6 +43,8 @@ abstract class SolrReindexBase implements SolrReindexHandler
|
||||
$taskName,
|
||||
$classes = null
|
||||
) {
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
// Filter classes for this index
|
||||
$indexClasses = $this->getClassesForIndex($indexInstance, $classes);
|
||||
|
||||
@ -53,10 +57,12 @@ abstract class SolrReindexBase implements SolrReindexHandler
|
||||
$includeSubclasses = $options['include_children'];
|
||||
|
||||
foreach (SearchVariant::reindex_states($class, $includeSubclasses) as $state) {
|
||||
if (!$searchableService->variantStateExcluded($state)) {
|
||||
$this->processVariant($logger, $indexInstance, $state, $class, $includeSubclasses, $batchSize, $taskName);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get valid classes and options for an index with an optional filter
|
||||
@ -238,13 +244,12 @@ abstract class SolrReindexBase implements SolrReindexHandler
|
||||
$items = $items->filter('ClassName', $class);
|
||||
}
|
||||
|
||||
$indexableService = IndexableService::singleton();
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
// ShowInSearch filter
|
||||
// we cannot use $items->remove($item), as that deletes the record from the database
|
||||
// Filter out objects that must not be indexed
|
||||
$idsToRemove = [];
|
||||
foreach ($items as $item) {
|
||||
if (!$indexableService->isIndexable($item)) {
|
||||
if (!$searchableService->isIndexable($item)) {
|
||||
$idsToRemove[] = $item->ID;
|
||||
}
|
||||
}
|
||||
|
||||
@ -9,7 +9,7 @@ use SilverStripe\FullTextSearch\Search\Indexes\SearchIndex;
|
||||
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
|
||||
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery_Range;
|
||||
use SilverStripe\FullTextSearch\Search\SearchIntrospection;
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariant;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariant_Caller;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\SolrService;
|
||||
@ -778,15 +778,14 @@ abstract class SolrIndex extends SearchIndex
|
||||
\Apache_Solr_Service::METHOD_POST
|
||||
);
|
||||
|
||||
$indexableService = IndexableService::singleton();
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
$results = new ArrayList();
|
||||
if ($res->getHttpStatus() >= 200 && $res->getHttpStatus() < 300) {
|
||||
foreach ($res->response->docs as $doc) {
|
||||
$result = DataObject::get_by_id($doc->ClassName, $doc->ID);
|
||||
if ($result) {
|
||||
// Filter out any results previously added to the solr index where ShowInSearch == false
|
||||
if (!$indexableService->isIndexable($result)) {
|
||||
if (!$searchableService->isViewable($result)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
|
||||
@ -7,6 +7,7 @@ use SilverStripe\Core\Config\Config;
|
||||
use SilverStripe\Core\Injector\Injector;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Tests\BatchedProcessorTest\BatchedProcessor_QueuedJobService;
|
||||
use SilverStripe\FullTextSearch\Tests\BatchedProcessorTest\BatchedProcessorTest_Index;
|
||||
use SilverStripe\FullTextSearch\Tests\BatchedProcessorTest\BatchedProcessorTest_Object;
|
||||
@ -120,6 +121,9 @@ class BatchedProcessorTest extends SapphireTest
|
||||
*/
|
||||
public function testBatching()
|
||||
{
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', [SiteTree::class]);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
$index = singleton(BatchedProcessorTest_Index::class);
|
||||
$index->reset();
|
||||
$processor = $this->generateDirtyIds();
|
||||
|
||||
@ -1,55 +0,0 @@
|
||||
<?php
|
||||
|
||||
namespace SilverStripe\FullTextSearch\Tests;
|
||||
|
||||
use SilverStripe\CMS\Model\SiteTree;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
|
||||
class IndexableServiceTest extends SapphireTest
|
||||
{
|
||||
protected $usesDatabase = true;
|
||||
|
||||
public function setup()
|
||||
{
|
||||
parent::setup();
|
||||
IndexableService::singleton()->clearCache();
|
||||
}
|
||||
|
||||
public function testIsIndexable()
|
||||
{
|
||||
$indexableService = IndexableService::singleton();
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertTrue($indexableService->isIndexable($page));
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 0;
|
||||
$page->write();
|
||||
$this->assertFalse($indexableService->isIndexable($page));
|
||||
}
|
||||
|
||||
public function testClearCache()
|
||||
{
|
||||
$indexableService = IndexableService::singleton();
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 0;
|
||||
$page->write();
|
||||
$this->assertFalse($indexableService->isIndexable($page));
|
||||
|
||||
// test the results are cached (expect stale result)
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertFalse($indexableService->isIndexable($page));
|
||||
|
||||
// after clearing cache, expect fresh result
|
||||
$indexableService->clearCache();
|
||||
$this->assertTrue($indexableService->isIndexable($page));
|
||||
}
|
||||
}
|
||||
@ -8,6 +8,7 @@ use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Processors\SearchUpdateProcessor;
|
||||
use SilverStripe\FullTextSearch\Search\Processors\SearchUpdateImmediateProcessor;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchUpdaterTest\SearchUpdaterTest_Container;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchUpdaterTest\SearchUpdaterTest_HasOne;
|
||||
@ -49,6 +50,9 @@ class SearchUpdaterTest extends SapphireTest
|
||||
|
||||
public function testHasOneHook()
|
||||
{
|
||||
$classesToSkip = [SearchUpdaterTest_Container::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$hasOne = new SearchUpdaterTest_HasOne();
|
||||
$hasOne->write();
|
||||
|
||||
@ -127,6 +131,9 @@ class SearchUpdaterTest extends SapphireTest
|
||||
|
||||
public function testHasManyHook()
|
||||
{
|
||||
$classesToSkip = [SearchUpdaterTest_Container::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$container1 = new SearchUpdaterTest_Container();
|
||||
$container1->write();
|
||||
|
||||
|
||||
@ -7,6 +7,7 @@ use SilverStripe\CMS\Model\SiteTree;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Indexes\SearchIndex_Recording;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchVariantVersionedTest\SearchVariantVersionedTest_Index;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchVariantVersionedTest\SearchVariantVersionedTest_Item;
|
||||
@ -45,6 +46,9 @@ class SearchVariantVersionedTest extends SapphireTest
|
||||
public function testPublishing()
|
||||
{
|
||||
// Check that write updates Stage
|
||||
$classesToSkip = [SearchVariantVersionedTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
$item = new SearchVariantVersionedTest_Item(array('TestText' => 'Foo'));
|
||||
$item->write();
|
||||
|
||||
120
tests/SearchableServiceTest.php
Normal file
120
tests/SearchableServiceTest.php
Normal file
@ -0,0 +1,120 @@
|
||||
<?php
|
||||
|
||||
namespace SilverStripe\FullTextSearch\Tests;
|
||||
|
||||
use SilverStripe\CMS\Model\SiteTree;
|
||||
use SilverStripe\Core\Config\Config;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
use SilverStripe\Security\Member;
|
||||
use SilverStripe\Versioned\Versioned;
|
||||
|
||||
class SearchableServiceTest extends SapphireTest
|
||||
{
|
||||
|
||||
protected $usesDatabase = true;
|
||||
|
||||
public function setup()
|
||||
{
|
||||
parent::setup();
|
||||
SearchableService::singleton()->clearCache();
|
||||
}
|
||||
|
||||
public function testIsIndexable()
|
||||
{
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', [SiteTree::class]);
|
||||
|
||||
Member::actAs(null, function () {
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertTrue($searchableService->isIndexable($page));
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 0;
|
||||
$page->write();
|
||||
$this->assertFalse($searchableService->isIndexable($page));
|
||||
});
|
||||
}
|
||||
|
||||
public function testIsViewable()
|
||||
{
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
|
||||
Member::actAs(null, function () {
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertTrue($searchableService->isViewable($page));
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'LoggedInUsers';
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertFalse($searchableService->isViewable($page));
|
||||
});
|
||||
}
|
||||
|
||||
public function testClearCache()
|
||||
{
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', [SiteTree::class]);
|
||||
|
||||
$searchableService = SearchableService::singleton();
|
||||
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'Anyone';
|
||||
$page->ShowInSearch = 0;
|
||||
$page->write();
|
||||
$this->assertFalse($searchableService->isIndexable($page));
|
||||
|
||||
// test the results are cached (expect stale result)
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertFalse($searchableService->isIndexable($page));
|
||||
|
||||
// after clearing cache, expect fresh result
|
||||
$searchableService->clearCache();
|
||||
$this->assertTrue($searchableService->isIndexable($page));
|
||||
}
|
||||
|
||||
public function testSkipIndexingCanViewCheck()
|
||||
{
|
||||
$searchableService = SearchableService::singleton();
|
||||
$page = SiteTree::create();
|
||||
$page->CanViewType = 'LoggedInUsers';
|
||||
$page->ShowInSearch = 1;
|
||||
$page->write();
|
||||
$this->assertFalse($searchableService->isIndexable($page));
|
||||
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', [SiteTree::class]);
|
||||
$searchableService->clearCache();
|
||||
$this->assertTrue($searchableService->isIndexable($page));
|
||||
}
|
||||
|
||||
public function testVariantStateExcluded()
|
||||
{
|
||||
$searchableService = SearchableService::singleton();
|
||||
$variantStateDraft = [SearchVariantVersioned::class => Versioned::DRAFT];
|
||||
$variantStateLive = [SearchVariantVersioned::class => Versioned::LIVE];
|
||||
|
||||
// default variant_state_draft_excluded = true
|
||||
$this->assertTrue($searchableService->variantStateExcluded($variantStateDraft));
|
||||
$this->assertFalse($searchableService->variantStateExcluded($variantStateLive));
|
||||
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
$this->assertFalse($searchableService->variantStateExcluded($variantStateDraft));
|
||||
$this->assertFalse($searchableService->variantStateExcluded($variantStateLive));
|
||||
}
|
||||
}
|
||||
@ -13,6 +13,7 @@ use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Processors\SearchUpdateImmediateProcessor;
|
||||
use SilverStripe\FullTextSearch\Search\Processors\SearchUpdateProcessor;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantSubsites;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\Solr4Service;
|
||||
@ -107,6 +108,10 @@ class SolrIndexSubsitesTest extends SapphireTest
|
||||
|
||||
public function testPublishing()
|
||||
{
|
||||
$classesToSkip = [SiteTree::class, File::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
// Setup mocks
|
||||
$serviceMock = $this->getServiceMock();
|
||||
self::$index->setService($serviceMock);
|
||||
|
||||
@ -4,7 +4,6 @@ namespace SilverStripe\FullTextSearch\Tests;
|
||||
|
||||
use Apache_Solr_Document;
|
||||
use Page;
|
||||
use SebastianBergmann\Version;
|
||||
use SilverStripe\Assets\File;
|
||||
use SilverStripe\CMS\Model\SiteTree;
|
||||
use SilverStripe\Core\Config\Config;
|
||||
@ -14,10 +13,9 @@ use SilverStripe\Core\Kernel;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Queries\SearchQuery;
|
||||
use SilverStripe\FullTextSearch\Search\Services\IndexableService;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantSubsites;
|
||||
use SilverStripe\FullTextSearch\Solr\SolrIndex;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\Solr3Service;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\Solr4Service;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchUpdaterTest\SearchUpdaterTest_Container;
|
||||
@ -34,7 +32,6 @@ use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_ShowInSearchIn
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyPage;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyDataObjectOne;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyDataObjectTwo;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\Subsites\Model\Subsite;
|
||||
use SilverStripe\Versioned\Versioned;
|
||||
|
||||
@ -409,8 +406,10 @@ class SolrIndexTest extends SapphireTest
|
||||
*/
|
||||
public function testShowInSearch()
|
||||
{
|
||||
$defaultMode = Versioned::get_reading_mode();
|
||||
// allow anonymous users to assess draft-only content to pass canView() check (will auto-reset for next test)
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
$serviceMock = $this->getMockBuilder(Solr4Service::class)
|
||||
->setMethods(['addDocument', 'deleteById'])
|
||||
@ -508,10 +507,106 @@ class SolrIndexTest extends SapphireTest
|
||||
})]
|
||||
);
|
||||
|
||||
IndexableService::singleton()->clearCache();
|
||||
SearchableService::singleton()->clearCache();
|
||||
SearchUpdater::flush_dirty_indexes();
|
||||
}
|
||||
|
||||
/**
|
||||
* Test that canView() check is used to exclude DataObjects from being added to the index
|
||||
*
|
||||
* Note: this code path that really being tested here is SearchUpdateProcessor->prepareIndexes()
|
||||
* This code path is used for 'inlet' filtering on CMS->save()
|
||||
* The results of this will show-up in SolrIndex->_addAs()
|
||||
*/
|
||||
public function testCanView()
|
||||
{
|
||||
// allow anonymous users to assess draft-only content to pass canView() check (will auto-reset for next test)
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
$serviceMock = $this->getMockBuilder(Solr4Service::class)
|
||||
->setMethods(['addDocument', 'deleteById'])
|
||||
->getMock();
|
||||
|
||||
$index = new SolrIndexTest_ShowInSearchIndex();
|
||||
$index->setService($serviceMock);
|
||||
FullTextSearch::force_index_list($index);
|
||||
|
||||
// will get added
|
||||
$pageA = new Page();
|
||||
$pageA->Title = 'Test Page Anyone';
|
||||
$pageA->CanViewType = 'Anyone';
|
||||
$pageA->write();
|
||||
|
||||
// will get filtered out
|
||||
$page = new Page();
|
||||
$page->Title = 'Test Page LoggedInUsers';
|
||||
$page->CanViewType = 'LoggedInUsers';
|
||||
$page->write();
|
||||
|
||||
// will get added
|
||||
$fileA = new File();
|
||||
$fileA->Title = 'Test File Anyone';
|
||||
$fileA->CanViewType = 'Anyone';
|
||||
$fileA->write();
|
||||
|
||||
// will get filtered out
|
||||
$file = new File();
|
||||
$file->Title = 'Test File LoggedInUsers';
|
||||
$file->CanViewType = 'LoggedInUsers';
|
||||
$file->write();
|
||||
|
||||
// will get added
|
||||
$objOneA = new SolrIndexTest_MyDataObjectOne();
|
||||
$objOneA->Title = 'Test MyDataObjectOne true';
|
||||
$objOneA->ShowInSearch = true;
|
||||
$objOneA->CanViewValue = true;
|
||||
$objOneA->write();
|
||||
|
||||
// will get filtered out
|
||||
$objOne = new SolrIndexTest_MyDataObjectOne();
|
||||
$objOne->Title = 'Test MyDataObjectOne false';
|
||||
$objOne->ShowInSearch = true;
|
||||
$objOne->CanViewValue = false;
|
||||
$objOne->write();
|
||||
|
||||
$callback = function (Apache_Solr_Document $doc) use ($pageA, $fileA, $objOneA): bool {
|
||||
$validKeys = [
|
||||
Page::class . $pageA->ID,
|
||||
File::class . $fileA->ID,
|
||||
SolrIndexTest_MyDataObjectOne::class . $objOneA->ID
|
||||
];
|
||||
return in_array($this->createSolrDocKey($doc), $validKeys);
|
||||
};
|
||||
|
||||
$serviceMock
|
||||
->expects($this->exactly(3))
|
||||
->method('addDocument')
|
||||
->withConsecutive(
|
||||
[$this->callback($callback)],
|
||||
[$this->callback($callback)],
|
||||
[$this->callback($callback)]
|
||||
);
|
||||
|
||||
// This is what actually triggers all the solr stuff
|
||||
SearchUpdater::flush_dirty_indexes();
|
||||
|
||||
Versioned::set_reading_mode($defaultMode);
|
||||
// delete a solr doc by setting ShowInSearch to false
|
||||
$pageA->ShowInSearch = false;
|
||||
$pageA->write();
|
||||
|
||||
$serviceMock
|
||||
->expects($this->exactly(1))
|
||||
->method('deleteById')
|
||||
->withConsecutive(
|
||||
[$this->callback(function (string $docID) use ($pageA): bool {
|
||||
return strpos($docID, $pageA->ID . '-' . SiteTree::class) !== false;
|
||||
})]
|
||||
);
|
||||
|
||||
SearchableService::singleton()->clearCache();
|
||||
SearchUpdater::flush_dirty_indexes();
|
||||
}
|
||||
|
||||
protected function createSolrDocKey(Apache_Solr_Document $doc)
|
||||
|
||||
@ -9,8 +9,14 @@ class SolrIndexTest_MyDataObjectOne extends DataObject implements TestOnly
|
||||
{
|
||||
private static $db = [
|
||||
'Title' => 'Varchar(255)',
|
||||
'ShowInSearch' => 'Boolean'
|
||||
'ShowInSearch' => 'Boolean',
|
||||
'CanViewValue' => 'Boolean(true)',
|
||||
];
|
||||
|
||||
private static $table_name = 'SolrIndexTestMyDataObjectOne';
|
||||
|
||||
public function canView($member = null)
|
||||
{
|
||||
return $this->CanViewValue;
|
||||
}
|
||||
}
|
||||
|
||||
@ -6,6 +6,7 @@ use Apache_Solr_Document;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\Core\Config\Config;
|
||||
use SilverStripe\Core\Injector\Injector;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\ORM\DataObject;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\SearchIntrospection;
|
||||
@ -112,6 +113,10 @@ class SolrIndexVersionedTest extends SapphireTest
|
||||
|
||||
public function testPublishing()
|
||||
{
|
||||
$classesToSkip = [SearchVariantVersionedTest_Item::class, SolrIndexVersionedTest_Object::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
// Check that write updates Stage
|
||||
Versioned::set_stage(Versioned::DRAFT);
|
||||
|
||||
@ -166,6 +171,10 @@ class SolrIndexVersionedTest extends SapphireTest
|
||||
|
||||
public function testDelete()
|
||||
{
|
||||
$classesToSkip = [SearchVariantVersionedTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
// Delete the live record (not the stage)
|
||||
Versioned::set_stage(Versioned::DRAFT);
|
||||
|
||||
|
||||
@ -6,12 +6,14 @@ use SilverStripe\Core\Config\Config;
|
||||
use SilverStripe\Core\Injector\Injector;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Solr\Reindex\Handlers\SolrReindexHandler;
|
||||
use SilverStripe\FullTextSearch\Solr\Reindex\Handlers\SolrReindexQueuedHandler;
|
||||
use SilverStripe\FullTextSearch\Solr\Reindex\Jobs\SolrReindexGroupQueuedJob;
|
||||
use SilverStripe\FullTextSearch\Solr\Reindex\Jobs\SolrReindexQueuedJob;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\Solr4Service;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\SolrService;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchVariantVersionedTest\SearchVariantVersionedTest_Item;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrReindexQueuedTest\SolrReindexQueuedTest_Service;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrReindexTest\SolrReindexTest_Index;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrReindexTest\SolrReindexTest_Item;
|
||||
@ -135,6 +137,9 @@ class SolrReindexQueuedTest extends SapphireTest
|
||||
*/
|
||||
public function testReindexSegmentsGroups()
|
||||
{
|
||||
$classesToSkip = [SolrReindexTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$this->createDummyData(18);
|
||||
|
||||
// Deletes are performed in the main task prior to individual groups being processed
|
||||
@ -194,6 +199,9 @@ class SolrReindexQueuedTest extends SapphireTest
|
||||
*/
|
||||
public function testRunGroup()
|
||||
{
|
||||
$classesToSkip = [SolrReindexTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$this->createDummyData(18);
|
||||
|
||||
// Just do what the SolrReindexQueuedJob would do to create each sub
|
||||
|
||||
@ -10,6 +10,7 @@ use SilverStripe\Core\Config\Config;
|
||||
use SilverStripe\Core\Injector\Injector;
|
||||
use SilverStripe\Dev\SapphireTest;
|
||||
use SilverStripe\FullTextSearch\Search\FullTextSearch;
|
||||
use SilverStripe\FullTextSearch\Search\Services\SearchableService;
|
||||
use SilverStripe\FullTextSearch\Search\Updaters\SearchUpdater;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariant;
|
||||
use SilverStripe\FullTextSearch\Search\Variants\SearchVariantVersioned;
|
||||
@ -18,6 +19,7 @@ use SilverStripe\FullTextSearch\Solr\Reindex\Handlers\SolrReindexImmediateHandle
|
||||
use SilverStripe\FullTextSearch\Solr\Services\Solr4Service;
|
||||
use SilverStripe\FullTextSearch\Solr\Services\SolrService;
|
||||
use SilverStripe\FullTextSearch\Solr\Tasks\Solr_Reindex;
|
||||
use SilverStripe\FullTextSearch\Tests\SearchVariantVersionedTest\SearchVariantVersionedTest_Item;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyDataObjectOne;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyDataObjectTwo;
|
||||
use SilverStripe\FullTextSearch\Tests\SolrIndexTest\SolrIndexTest_MyPage;
|
||||
@ -235,6 +237,9 @@ class SolrReindexTest extends SapphireTest
|
||||
*/
|
||||
public function testRunGroup()
|
||||
{
|
||||
$classesToSkip = [SolrReindexTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$this->service->method('deleteByQuery')
|
||||
->with('+(ClassHierarchy:' . SolrReindexTest_Item::class . ') +_query_:"{!frange l=2 u=2}mod(ID, 6)" +(_testvariant:"1")');
|
||||
|
||||
@ -266,6 +271,9 @@ class SolrReindexTest extends SapphireTest
|
||||
*/
|
||||
public function testRunAllGroups()
|
||||
{
|
||||
$classesToSkip = [SolrReindexTest_Item::class];
|
||||
Config::modify()->set(SearchableService::class, 'indexing_canview_exclude_classes', $classesToSkip);
|
||||
|
||||
$this->service->method('deleteByQuery')
|
||||
->withConsecutive(
|
||||
['+(ClassHierarchy:' . SolrReindexTest_Item::class . ') +_query_:"{!frange l=0 u=0}mod(ID, 6)" +(_testvariant:"1")'],
|
||||
@ -305,8 +313,10 @@ class SolrReindexTest extends SapphireTest
|
||||
*/
|
||||
public function testShowInSearch()
|
||||
{
|
||||
$defaultMode = Versioned::get_reading_mode();
|
||||
// allow anonymous users to assess draft-only content to pass canView() check (will auto-reset for next test)
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
// will get added
|
||||
$pageA = new Page();
|
||||
@ -394,8 +404,89 @@ class SolrReindexTest extends SapphireTest
|
||||
$handler->runGroup($logger, $index, $state, SiteTree::class, 1, 0);
|
||||
$handler->runGroup($logger, $index, $state, File::class, 1, 0);
|
||||
$handler->runGroup($logger, $index, $state, SolrIndexTest_MyDataObjectOne::class, 1, 0);
|
||||
}
|
||||
|
||||
Versioned::set_reading_mode($defaultMode);
|
||||
/**
|
||||
* Test that CanView filtering is working correctly
|
||||
*/
|
||||
public function testCanView()
|
||||
{
|
||||
// allow anonymous users to assess draft-only content to pass canView() check (will auto-reset for next test)
|
||||
Versioned::set_draft_site_secured(false);
|
||||
Versioned::set_reading_mode('Stage.' . Versioned::DRAFT);
|
||||
Config::modify()->set(SearchableService::class, 'variant_state_draft_excluded', false);
|
||||
|
||||
// will get added
|
||||
$pageA = new Page();
|
||||
$pageA->Title = 'Test Page Anyone';
|
||||
$pageA->CanViewType = 'Anyone';
|
||||
$pageA->write();
|
||||
|
||||
// will get filtered out
|
||||
$page = new Page();
|
||||
$page->Title = 'Test Page LoggedInUsers';
|
||||
$page->CanViewType = 'LoggedInUsers';
|
||||
$page->write();
|
||||
|
||||
// will get added
|
||||
$fileA = new File();
|
||||
$fileA->Title = 'Test File Anyone';
|
||||
$fileA->CanViewType = 'Anyone';
|
||||
$fileA->write();
|
||||
|
||||
// will get filtered out
|
||||
$file = new File();
|
||||
$file->Title = 'Test File LoggedInUsers';
|
||||
$file->CanViewType = 'LoggedInUsers';
|
||||
$file->write();
|
||||
|
||||
// will get added
|
||||
$objOneA = new SolrIndexTest_MyDataObjectOne();
|
||||
$objOneA->Title = 'Test MyDataObjectOne true';
|
||||
$objOneA->CanViewValue = true;
|
||||
$objOneA->ShowInSearch = true;
|
||||
$objOneA->write();
|
||||
|
||||
// will get filtered out
|
||||
$objOne = new SolrIndexTest_MyDataObjectOne();
|
||||
$objOne->Title = 'Test MyDataObjectOne false';
|
||||
$objOne->CanViewValue = false;
|
||||
$objOneA->ShowInSearch = true;
|
||||
$objOne->write();
|
||||
|
||||
$serviceMock = $this->getMockBuilder(Solr4Service::class)
|
||||
->setMethods(['addDocument', 'deleteByQuery'])
|
||||
->getMock();
|
||||
|
||||
$index = new SolrIndexTest_ShowInSearchIndex();
|
||||
$index->setService($serviceMock);
|
||||
FullTextSearch::force_index_list($index);
|
||||
|
||||
$callback = function (Apache_Solr_Document $doc) use ($pageA, $fileA, $objOneA): bool {
|
||||
$validKeys = [
|
||||
Page::class . $pageA->ID,
|
||||
File::class . $fileA->ID,
|
||||
SolrIndexTest_MyDataObjectOne::class . $objOneA->ID,
|
||||
];
|
||||
$solrDocKey = $this->createSolrDocKey($doc);
|
||||
return in_array($this->createSolrDocKey($doc), $validKeys);
|
||||
};
|
||||
|
||||
$serviceMock
|
||||
->expects($this->exactly(3))
|
||||
->method('addDocument')
|
||||
->withConsecutive(
|
||||
[$this->callback($callback)],
|
||||
[$this->callback($callback)],
|
||||
[$this->callback($callback)]
|
||||
);
|
||||
|
||||
$logger = new SolrReindexTest_RecordingLogger();
|
||||
$state = [SearchVariantVersioned::class => Versioned::DRAFT];
|
||||
$handler = Injector::inst()->get(SolrReindexImmediateHandler::class);
|
||||
$handler->runGroup($logger, $index, $state, SiteTree::class, 1, 0);
|
||||
$handler->runGroup($logger, $index, $state, File::class, 1, 0);
|
||||
$handler->runGroup($logger, $index, $state, SolrIndexTest_MyDataObjectOne::class, 1, 0);
|
||||
}
|
||||
|
||||
protected function createSolrDocKey(Apache_Solr_Document $doc)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user