Unexpected output: Affected paths: relative - resubmit patch.
⚙ D343 INP-1744 Make page content blocks searcheable
Page MenuHomeIn-Portal Phabricator
Differential D343

INP-1744 Make page content blocks searcheable
ClosedPublic
Actions

Authored by erik on May 11 2018, 4:16 AM.

Details

Reviewers
alex
glebs
Commits
rINP16634: Fixes INP-1744 - Make page content blocks searcheable
rTDEF16635: Bug INP-1744 - Make page content blocks searcheable
rTSIM16636: Bug INP-1744 - Make page content blocks searcheable
Test Plan

Test upgrade

  1. don't apply the patch
  2. in home page's content block enter any text that is 5KB large or longer with unique word at the end
  3. perform a search using that unique word
  4. confirm, that nothing is found
  5. apply the patch
  6. perform an upgrade to In-Portal v5.2.2-B3
  7. perform a search using that unique word
  8. confirm, that homepage is found

Test clean install

  1. apply the patch (if wasn't applied yet)
  2. perform clean install of In-Portal v5.2.2-B3
  3. in home page's content block enter any text that is 5KB large or longer with unique word at the end
  4. perform a search using that unique word
  5. confirm, that homepage is found

Test search setting

  1. login into Admin Console
  2. go to Configuration → Website → General section
  3. change Admin Console Interface setting to advanced and save changes
  4. go to Configuration → Website → Search section
  5. confirm, that Page Content row is found there and it's enabled for both simple and advanced search

Test search in "default" theme with Demo Data

  1. perform clean install WITHOUT MODULES using default theme with Install Demo content for this theme checkbox checked
  2. login to Admin Console
  3. go to Configuration → Website → Themes section
  4. enable advanced theme (don't make it as primary)
  5. open website using advanced theme (e.g. .../advanced/ URL)
  6. search for "dust-free"
  7. confirm, that "Services" page was found

Test search in "simple" theme with Demo Data

  1. perform clean install WITHOUT MODULES using simple theme with Install Demo content for this theme checkbox checked
  2. login to Admin Console
  3. go to Configuration → Website → Themes section
  4. enable advanced theme (don't make it as primary)
  5. open website using advanced theme (e.g. .../advanced/ URL)
  6. search for "dust-free"
  7. confirm, that "Services" page was found

Diff Detail

Repository
rINP In-Portal
Branch
/in-portal/branches/5.2.x
Lint
Lint ErrorsExcuse: Unable to fix CS issues without massive indentation changes.
SeverityLocationCodeMessage
Errorcore/install/upgrades.php:2368PHPCS.E.CodingStandard.Commenting.FunctionComment.ScalarTypeHintMissingCodingStandard.Commenting.FunctionComment.ScalarTypeHintMissing
Errorcore/install/upgrades.php:2368PHPCS.E.CodingStandard.NamingConventions.ValidFunctionName.ScopeNotCamelCapsCodingStandard.NamingConventions.ValidFunctionName.ScopeNotCamelCaps
Errorcore/install/upgrades.php:2368PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/install/upgrades.php:2370PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/install/upgrades.php:2372PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/install/upgrades.php:2385PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/install/upgrades.php:2387PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/install/upgrades.php:2388PHPCS.E.Generic.WhiteSpace.ScopeIndent.IncorrectExactGeneric.WhiteSpace.ScopeIndent.IncorrectExact
Errorcore/units/categories/categories_config.php:458PHPCS.E.CodingStandard.Commenting.InlineComment.InvalidEndCharCodingStandard.Commenting.InlineComment.InvalidEndChar
Errorcore/units/categories/categories_config.php:458PHPCS.E.CodingStandard.Commenting.InlineComment.NotCapitalCodingStandard.Commenting.InlineComment.NotCapital
Errorcore/units/helpers/page_helper.php:449PHPCS.E.CodingStandard.Commenting.FunctionComment.ScalarTypeHintMissingCodingStandard.Commenting.FunctionComment.ScalarTypeHintMissing
Errorcore/units/helpers/page_helper.php:482PHPCS.E.CodingStandard.Commenting.FunctionComment.ScalarTypeHintMissingCodingStandard.Commenting.FunctionComment.ScalarTypeHintMissing
Errorcore/units/page_revisions/page_revisions_config.php:89PHPCS.E.CodingStandard.Array.Array.SpaceAfterKeywordCodingStandard.Array.Array.SpaceAfterKeyword
Errorcore/units/page_revisions/page_revisions_config.php:89PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/page_revisions/page_revisions_config.php:95PHPCS.E.CodingStandard.Array.Array.SpaceAfterKeywordCodingStandard.Array.Array.SpaceAfterKeyword
Errorcore/units/page_revisions/page_revisions_config.php:95PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Unit
No Unit Test Coverage
Build Status
Buildable 903
Build 903: arc lint + arc unit

Event Timeline

glebs created this revision.May 11 2018, 4:16 AM
glebs edited the test plan for this revision. (Show Details)May 11 2018, 4:20 AM
glebs planned changes to this revision.May 11 2018, 5:06 AM
glebs updated this revision to Diff 856.May 11 2018, 6:01 AM

Requested changes

glebs edited the test plan for this revision. (Show Details)May 11 2018, 6:05 AM
glebs edited reviewers, added: erik; removed: alex.
glebs edited the test plan for this revision. (Show Details)
glebs planned changes to this revision.May 11 2018, 7:38 AM
glebs updated this revision to Diff 857.May 11 2018, 7:42 AM

add charset parameter to html_entity_decode in \PageHelper::makeSearchable

erik accepted this revision.May 11 2018, 9:17 AM
This revision is now accepted and ready to land.May 11 2018, 9:17 AM
alex requested changes to this revision.Oct 25 2018, 3:47 AM
alex added a subscriber: alex.
alex added inline comments.
core/units/helpers/page_helper.php
455–464

Instead of selecting single row, where content block data is merged per-language we need to:

  1. select all content block data for given revision
  2. manually merge data using PHP (merge all l1_Content columns together as l1_PageContent, then merge all l2_Content columns together as l2_PageContent and so on)

The GROUP_CONCAT function truncates result to 1024 bytes (see https://dev.mysql.com/doc/refman/8.0/en/group-by-functions.html#function_group-concat) making resulting string only 1KB long. This prevents searching pages with more than 1KB long.

This revision now requires changes to proceed.Oct 25 2018, 3:47 AM
alex added a comment.Oct 25 2018, 3:48 AM

Also update test plan to cover cases, when we're searching last word in large text (more then 65 KB).

erik edited the test plan for this revision. (Show Details)Oct 26 2018, 6:40 AM
erik updated this revision to Diff 871.Oct 26 2018, 6:46 AM
erik edited edge metadata.

GROUP_CONCAT usage replaced with concatenation on php level.

alex requested changes to this revision.Oct 26 2018, 7:54 AM

Test plan not updated to cover searching through large amount of text.

This revision now requires changes to proceed.Oct 26 2018, 7:54 AM
erik commandeered this revision.Oct 26 2018, 8:19 AM
erik requested review of this revision.
erik edited reviewers, added: glebs; removed: erik.
erik removed a reviewer: erik.
erik edited edge metadata.
alex edited the test plan for this revision. (Show Details)Oct 26 2018, 8:25 AM
alex requested changes to this revision.Oct 26 2018, 8:43 AM

No matter how I try to follow test plan I'm get getting pages used in test:

  • if installing advanced theme without/without test data I'm not getting About page
  • if installing default theme with test data, then I'm getting About page, but there is no search box in default theme
  • if installing default theme with test data and then switching to advanced theme, then I'm getting About page and test data, but searching gadget returns no results, because we're not populating new PageRevisions.l1_PageContent column for revisions created from install_data.sql files with themes (the demo data)

Things to do (except for adding missing phrase):

  1. update install_data.sql files for default and simple themes so that new columns are being populated correctly
  2. update test plan to actually explain how to get desired page content in the first place
  3. add test case, when administrator is changing content block content (e.g. adding new text) and that new text immediately becomes searcheable
core/install/install_data.sql
997

Phrase lc_field_PageContent used here isn't translated.

This revision now requires changes to proceed.Oct 26 2018, 8:43 AM
erik updated this revision to Diff 872.Oct 26 2018, 10:11 AM
erik edited edge metadata.

Added missing phrase translation.

alex edited the test plan for this revision. (Show Details)Nov 12 2018, 3:38 AM
alex added a project: Restricted Project.Nov 12 2018, 3:41 AM
alex updated this revision to Diff 883.EditedNov 12 2018, 3:49 AM
  1. Updated demo data of "default" and "simple" themes to make them searchable immediately after installation.
  2. Improved trimming logic, when combining content block data to end up in less trailing spaces.
  3. Fixed notice during search about missing PageContent field of categories.
alex accepted this revision.Nov 13 2018, 3:23 AM
alex edited the test plan for this revision. (Show Details)
This revision is now accepted and ready to land.Nov 13 2018, 3:23 AM
Closed by commit rINP16634: Fixes INP-1744 - Make page content blocks searcheable (authored by alex). · Explain WhyNov 13 2018, 3:38 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

DiffIDBaseDescriptionCreatedLintUnit
BaseBase
Diff 185516632May 11 2018, 4:16 AM
★
★
Diff 285616632Requested changesMay 11 2018, 6:01 AM
★
★
Diff 385716632add charset parameter to `html_entity_decode` in `\PageHelper::makeSearchable`May 11 2018, 7:42 AM
★
★
Diff 487116632GROUP_CONCAT usage replaced with concatenation on php level.Oct 26 2018, 6:46 AM
★
★
Diff 587216632Added missing phrase translation.Oct 26 2018, 10:11 AM
★
★
Diff 6883166331. Updated demo data of "default" and "simple" themes to make them searchable…Nov 12 2018, 3:49 AM
★
★
Diff 788416633rINP16634Nov 13 2018, 3:38 AM
★
★

Diff 883

View Options

core/install/english.lang

Loading...
View Options

core/install/install_data.sql

Loading...
View Options

core/install/install_schema.sql

Loading...
View Options

core/install/upgrades.php

Loading...
View Options

core/install/upgrades.sql

Loading...
View Options

core/units/categories/categories_config.php

Loading...
View Options

core/units/helpers/page_helper.php

Loading...
View Options

core/units/page_revisions/page_revision_eh.php

Loading...
View Options

core/units/page_revisions/page_revisions_config.php

Loading...
View Options

themes/default/_install/install_data.sql

Loading...
View Options

themes/simple/_install/install_data.sql

Loading...
In-Portal CMS · In-Portal Community Pages · In-Portal Issue Tracker · In-Portal Repository Browser · Quality Assurance · In-Portal CI Server