Page MenuHomeIn-Portal Phabricator

INP-1854 - Create global switch for Web Request/Session Data logging into the System Log
AcceptedPublic

Authored by alex on Jan 24 2024, 6:19 AM.

Details

Reviewers
erik
Test Plan
NOTE: Apply patches from the D464 and D456 (if absent) before conditing the testing.

Part 1 (Clean Install)

  • in Web Browser:
    1. peform a clean install
    2. while on the System Configuration installation step confirm, that:
      • the Include Web Request data in "System Log" setting is visible and is enabled
      • the Largest Web Request/Session Size, that will be saved in "System Log" setting is visible and has 5MB value
      • disable the Include Web Request data in "System Log" setting
      • set 5KB in the Largest Web Request/Session Size, that will be saved in "System Log" setting
      • proceed to the next step
  • in the IDE:
    1. open the /system/config.php file
    2. confirm, that it has these lines:
$_CONFIG['Misc']['SystemLogIncludeRequestData'] = '0';
$_CONFIG['Misc']['SystemLogRequestDataLimit'] = '5KB';

Part 2 (Upgrade)

  • in Web Browser:
    1. upgrade from any previous In-Portal version (e.g. In-Portal 5.2.2-B1)
    2. while on the System Configuration installation step confirm, that:
      • the Include Web Request data in "System Log" setting is visible and is enabled
      • the Largest Web Request/Session Size, that will be saved in "System Log" setting is visible and has 5MB value
      • disable the Include Web Request data in "System Log" setting
      • set 5KB in the Largest Web Request/Session Size, that will be saved in "System Log" setting
      • proceed to the next step
  • in the IDE:
    1. open the /system/config.php file
    2. confirm, that it has these lines:
$_CONFIG['Misc']['SystemLogIncludeRequestData'] = '0';
$_CONFIG['Misc']['SystemLogRequestDataLimit'] = '5KB';
  • in Web Browser:
    1. login to Admin Console
    2. go to the ConfigurationWebsiteAdvanced section
    3. confirm, that:
      • the Keep Request Data in "System Log" for system setting is present
      • has value of 1 month set in the dropdown

Part 3 (Setting change)

  1. login to Admin Console
  2. go to the ConfigurationWebsiteAdvanced section
  3. confirm, that:
    • the Keep Request Data in "System Log" for system setting is present
    • has value of 1 month set in the dropdown
  4. set value for the Keep "System Log" for system setting to 1 month
  5. set value for the Keep Request Data in "System Log" for system setting to 2 weeks
  6. save changes
  7. confirm, that changes were saved
  8. set value for the Keep Request Data in "System Log" for system setting to 3 months
  9. save changes
  10. confirm, that changes were not saved and the error message was shown

Part 4 (Adding Fixtures)

  • in IDE
    1. open the `` file for editing
    2. add this code at the beginning of the \UsersEventHandler::SetCustomQuery method:
$log = $this->Application->log('test message (default request/session data)', 123);
$log->setLogLevel(kLogger::LL_ERROR);
$log->write();

$log = $this->Application->log('test message (force add request data)', 124);
$log->setLogLevel(kLogger::LL_ERROR);
$log->addRequestData();
$log->write();

$log = $this->Application->log('test message (force remove request data)', 125);
$log->setLogLevel(kLogger::LL_ERROR);
$log->removeRequestData();
$log->write();

$log = $this->Application->log('test message (force add session data)', 126);
$log->setLogLevel(kLogger::LL_ERROR);
$log->addSessionData();
$log->write();

$log = $this->Application->log('test message (force remove session data)', 127);
$log->setLogLevel(kLogger::LL_ERROR);
$log->removeRequestData();
$log->write();

Part 5 (Auto Logging Disabled)

  • in IDE:
    1. open the /system/config.php file
    2. change SystemLogIncludeRequestData setting to 0
    3. change SystemLogRequestDataLimit setting to 5KB
    4. save changes
  • in the Admin Console:
    1. go to the User ManagementUsers section
    2. go to the Logs & ReportsSystem Log section
    3. confirm, that among 5 newly created records with following properties:
      • message: test message (default request/session data) + no request data + no session data
      • message: test message (force add request data) + request data + no session data
      • message: test message (force remove request data) + no request data + no session data
      • message: test message (force add session data) + no request data + session data
      • message: test message (force remove session data) + no request data + no session data

Part 6 (Auto Logging Without Limits)

  • in IDE:
    1. open the /system/config.php file
    2. change SystemLogIncludeRequestData setting to 1
    3. change SystemLogRequestDataLimit setting to 5MB
    4. save changes
  • in the Admin Console:
    1. go to the User ManagementUsers section
    2. go to the Logs & ReportsSystem Log section
    3. confirm, that among 5 newly created records with following properties:
      • message: test message (default request/session data) + request data + session data
      • message: test message (force add request data) + request data + session data
      • message: test message (force remove request data) + no request data + session data
      • message: test message (force add session data) + request data + session data
      • message: test message (force remove session data) + request data + no session data

Part 7 (Auto Logging With Limit)

  • in IDE:
    1. open the /system/config.php file
    2. change SystemLogIncludeRequestData setting to 1
    3. change SystemLogRequestDataLimit setting to 1500
    4. save changes
  • in the Admin Console:
    1. go to the User ManagementUsers section
    2. go to the Logs & ReportsSystem Log section
    3. this is cut off message format (used below): The %d bytes (%d elements) of data not logged, because it is larger, than %d bytes limit.
    4. confirm, that among 5 newly created records with following properties (when data is logged, then at least 1 of the request data/session data must have system-log-error array key with the cut off message similar to the above-shown):
      • message: test message (default request/session data) + request data + session data
      • message: test message (force add request data) + request data + session data
      • message: test message (force remove request data) + no request data + session data
      • message: test message (force add session data) + request data + session data
      • message: test message (force remove session data) + request data + no session data

Part 8 (Rotation)

  • in the database:
    1. open SystemLog table for editing
    2. pick any 2 records, that have both request data & session data present
    3. for 1 of the above records change LogTimestamp to be above 1 week from now on (the strtotime('-2 weeks'); PHP code will produce that timestamp)
  • in the Admin Console:
    1. go to the Logs & ReportsSystem Log section
    2. open 2 above-picked system log record for editing
    3. confirm, that they both have request data & session data
    4. go to the ConfigurationWebsiteAdvanced section
    5. set the Keep Request Data in "System Log" for system setting to the 1 week
    6. save changes
    7. go to the ConfigurationWebsiteScheduled Tasks section
    8. confirm, that the scheduled task with rotate_system_log_request_data name is present
    9. run the scheduled task with rotate_system_log_request_data name is present
    10. go to the Logs & ReportsSystem Log section
    11. confirm, that from 2 above-picked records:
      • record with newer LogTimestamp retained it's request data & session data
      • record with patched LogTimestamp lost it's request data & session data

Part 9 (CLI not logged)

  • in IDE:
    1. open the /system/config.php file
    2. change SystemLogIncludeRequestData setting to 1
    3. change SystemLogRequestDataLimit setting to 5MB
    4. save changes
    5. open the /tools/cron.php file
    6. replace the $application->EventManager->runScheduledTasks(true); line with the
$log = $application->log('test message (default request/session data + cron)', 221);
$log->setLogLevel(kLogger::LL_ERROR);
$log->write();
    1. save changes
  • in CLI:
    1. run the php tools/cron.php
  • in the Admin Console:
    1. go to the Logs & ReportsSystem Log section
    2. confirm, that system log record with test message (default request/session data + cron) message was created
    3. confirm, that the above record don't have (even empty array) request & session data present

Diff Detail

Repository
rINP In-Portal
Branch
/in-portal/branches/5.2.x
Lint
Lint ErrorsExcuse: New code is formatted according to code around it
SeverityLocationCodeMessage
Errorcore/kernel/utility/logger.php:273PHPCS.E.CodingStandard.WhiteSpace.ControlStructureSpacing.SpaceBeforeCloseBraceCodingStandard.WhiteSpace.ControlStructureSpacing.SpaceBeforeCloseBrace
Errorcore/kernel/utility/logger.php:273PHPCS.E.CodingStandard.WhiteSpace.ControlStructureSpacing.SpacingAfterOpenBraceCodingStandard.WhiteSpace.ControlStructureSpacing.SpacingAfterOpenBrace
Errorcore/units/logs/system_logs/system_logs_config.php:37PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:37PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/logs/system_logs/system_logs_config.php:38PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:38PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/logs/system_logs/system_logs_config.php:152PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:152PHPCS.E.Generic.PHP.LowerCaseConstant.FoundGeneric.PHP.LowerCaseConstant.Found
Errorcore/units/logs/system_logs/system_logs_config.php:152PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/logs/system_logs/system_logs_config.php:153PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:153PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/logs/system_logs/system_logs_config.php:156PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:156PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Errorcore/units/logs/system_logs/system_logs_config.php:159PHPCS.E.CodingStandard.Arrays.Array.SpaceAfterKeywordCodingStandard.Arrays.Array.SpaceAfterKeyword
Errorcore/units/logs/system_logs/system_logs_config.php:159PHPCS.E.Generic.PHP.LowerCaseKeyword.FoundGeneric.PHP.LowerCaseKeyword.Found
Unit
No Unit Test Coverage
Build Status
Buildable 1243
Build 1243: arc lint + arc unit

Event Timeline

alex created this revision.Jan 24 2024, 6:19 AM
alex requested review of this revision.Jan 24 2024, 6:19 AM
alex edited the test plan for this revision. (Show Details)Jan 24 2024, 6:21 AM
alex updated this revision to Diff 1199.Jan 24 2024, 6:23 AM

Removed unrelated change to the System column in the database schema.

erik accepted this revision.Jan 24 2024, 8:43 AM

All works right, only test plan must be corrected.

  1. need $log->removeSessiontData() in place of $log->removeRequestData() on the last message from 5 (force remove session data).
  2. need remove excessive words "is present" from "run the scheduled task with rotate_system_log_request_data name is present" test plan part
This revision is now accepted and ready to land.Jan 24 2024, 8:43 AM
alex updated this revision to Diff 1279.Sat, Nov 16, 12:43 PM

Actualized "/system/config.php", that is created upon build.