* Fix MAX_BUCKETS_MEM overflow
* Fix MAX_QUERY_BUF overflow
* Fix int overflow in IsBucketValid function
* Add missing newline
* Remove test for max value of pgsm_query_shared_buffer parameter
* Tune tests
* Cleanup
* Use int64 type instead of long long
This bug uncovered serious issues with how the data was being stored by PSGM.
So it require a complete redesign.
pg_stat_monitor now stores the data locally within the backend process's local
memory. The data is only stored when the query completes. This reduces the
number of lock acquisitions that were previously needed during various stages
of the execution. Also, this avoids data loss in case the current bucket
changes during execution. Also, the unavailability of jumble state during later
stages of executions was causing pg_stat_monitor to save non-normalized query.
This was a major problem as well.
pg_stat_monitor specific memory context is implemented. It is used for saving
data locally. The context memory callback helps us clear the locally saved data
so that we do not store it multiple times in the shared hash.
As part of this major rewrite, pgss reference in function and variable names
is changed to pgsm. Memory footprint for the entries is reduced, data types
are corrected where needed, and we've removed unused variables, functions and
macros.
This patch was mutually created by:
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
Co-authored-by: Muhammad Usama <muhammad.usama@percona.com>
As part of this PR, also updated regression test cases that are related to
following JIRA issues as well.
PG-354 pg_stat_monitor: Remove pg_stat_monitor_settings view
Now we not using pg_stat_monitor_settings view, due to this change majority of
TAP testcase requried output changes.
PG-558: Create test case to verify the function names and count in PGSM.
Added additional output file for SQL test case.
PG-554: Remove redundant expected output files from regression.
Removed unnecessary output files in TAP testcases where these were not needed.
Regression was failing in main branch after merging REL_1_STABLE. Changes to
stablise the regression are summarised below.
Updated following TAP testcase with 'where' clause for filtering GUC specific row
from settings view. Intention is to fetch only GUC value and test it.
1) t/001_settings_default.pl
2) t/002_settings_pgsm_track_planning.pl
3) t/003_settings_pgms_extract_comments.pl
4) t/004_settings_pgsm_track.pl
5) t/005_settings_pgsm_enable_query_plan.pl
6) t/006_settings_pgsm_overflow_target.pl
7) t/007_settings_pgsm_query_shared_buffer.pl
8) t/008_settings_pgsm_histogram_buckets.pl
9) t/009_settings_pgsm_histogram_max.pl
10) t/010_settings_pgsm_histogram_min.pl
11) t/011_settings_pgsm_bucket_time.pl
12) t/012_settings_pgsm_max_buckets.pl
13) t/013_settings_pgsm_normalized_query.pl
14) t/014_settings_pgsm_track_utility.pl
15) t/015_settings_pgsm_query_max_len.pl
16) t/016_settings_pgsm_max.pl
Updated following expected files for TAP testcase where testcase was updated
with 'where' clause to filter GUC specific row only from settings view.
1) t/expected/001_settings_default.out
2) t/expected/002_settings_pgsm_track.outanning.out
3) t/expected/003_settings_pgms_extract_comments.out
4) t/expected/004_settings_pgsm_track.out
5) t/expected/005_settings_pgsm_enable_query.out
6) t/expected/006_settings_pgsm_overflow_target.out
7) t/expected/007_settings_pgsm_query_shared_buffer.out
8) t/expected/008_settings_pgsm_histogram_buckets.out
9) t/expected/009_settings_pgsm_histogram_max.out
10) t/expected/010_settings_pgsm_histogram_min.out
11) t/expected/011_settings_pgsm_bucket_time.out
12) t/expected/012_settings_pgsm_max_buckets.out
13) t/expected/013_settings_pgsm_normalized_query.out
14) t/expected/014_settings_pgsm_track_utility.out
15) t/expected/015_settings_pgsm_query_max_len.out
16) t/expected/016_settings_pgsm_max.out
17) t/expected/017_execution_stats.out
Updated expected files for following sql based testcases due to changed default
value of pgsm_normalized_query.
1) error_insert_1.out
2) guc_1.out
3) top_query_1.out
Enabled GUC pgsm_normalized_query to 'yes' for following testcases.
1) t/024_check_timings.pl
2) t/025_compare_pgss.pl
3) t/026_shared_blocks.pl
Added the output file for histogram testcase to accomodate behaviour.
Changes suggested by Ibrar to make regression more modular (PG-440), are also
made part of regression so those these can used for future improvements.
Removed commented/unneeded steps from t/017_execution_stats.pl.
Updated testcase tags.sql and output file tags.out to accomodate enabling of
pgsm_extract_comments to yes.
Updated error_1.out output file with required output, it was overwritten in
one of previous commits.
Updated test condition (count of calls and bucket) in t/023_missing_queries.pl
as per comment from Ibrar in PG-461 where PGSM does not track the pg_sleep in
some cases.
There was no maximum limit set for the number of maximum histograms
bucket, which can lead to a crash in case of higher value.
PG-382: Adjust the maximum value for histogram buckets.
Fix the regression issue related to GUC and set the maximum
buckets value correctly.
PG-382: Adjust the maximum value for histogram buckets.
Fix the TAP test cases.
This commit brings following changes to this branch:
1) Port changes/additions TAP testing from main branch to this branch, under PG-292.
2) Changes to test cases due to GUCs change, under PG-331.
3) Call counts verfications, under PG-338.
4) Changes to github workflows to accomodate automation for TAP testing, under PG-343.
Artem Gavrilov
Hamid Akhtar
Naeem Akhter
Ibrar Ahmed
Naeem Akhter