* PG-606: New GUC required for enabling/disabling of pgsm_query_id calculation
Adds a new GUC pg_stat_monitor.pgsm_enable_pgsm_query_id to enable/disable
pgsm query id calculation. Apart from that patch also refactors the GUC-related
code to match PostgreSQL conventions.
Moreover, the commit also changes the pgsm_enable_overflow GUC to boolean
instead of enum.
This bug uncovered serious issues with how the data was being stored by PSGM.
So it require a complete redesign.
pg_stat_monitor now stores the data locally within the backend process's local
memory. The data is only stored when the query completes. This reduces the
number of lock acquisitions that were previously needed during various stages
of the execution. Also, this avoids data loss in case the current bucket
changes during execution. Also, the unavailability of jumble state during later
stages of executions was causing pg_stat_monitor to save non-normalized query.
This was a major problem as well.
pg_stat_monitor specific memory context is implemented. It is used for saving
data locally. The context memory callback helps us clear the locally saved data
so that we do not store it multiple times in the shared hash.
As part of this major rewrite, pgss reference in function and variable names
is changed to pgsm. Memory footprint for the entries is reduced, data types
are corrected where needed, and we've removed unused variables, functions and
macros.
This patch was mutually created by:
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
Co-authored-by: Muhammad Usama <muhammad.usama@percona.com>
As part of this PR, also updated regression test cases that are related to
following JIRA issues as well.
PG-354 pg_stat_monitor: Remove pg_stat_monitor_settings view
Now we not using pg_stat_monitor_settings view, due to this change majority of
TAP testcase requried output changes.
PG-558: Create test case to verify the function names and count in PGSM.
Added additional output file for SQL test case.
PG-554: Remove redundant expected output files from regression.
Removed unnecessary output files in TAP testcases where these were not needed.
Regression was failing in main branch after merging REL_1_STABLE. Changes to
stablise the regression are summarised below.
Updated following TAP testcase with 'where' clause for filtering GUC specific row
from settings view. Intention is to fetch only GUC value and test it.
1) t/001_settings_default.pl
2) t/002_settings_pgsm_track_planning.pl
3) t/003_settings_pgms_extract_comments.pl
4) t/004_settings_pgsm_track.pl
5) t/005_settings_pgsm_enable_query_plan.pl
6) t/006_settings_pgsm_overflow_target.pl
7) t/007_settings_pgsm_query_shared_buffer.pl
8) t/008_settings_pgsm_histogram_buckets.pl
9) t/009_settings_pgsm_histogram_max.pl
10) t/010_settings_pgsm_histogram_min.pl
11) t/011_settings_pgsm_bucket_time.pl
12) t/012_settings_pgsm_max_buckets.pl
13) t/013_settings_pgsm_normalized_query.pl
14) t/014_settings_pgsm_track_utility.pl
15) t/015_settings_pgsm_query_max_len.pl
16) t/016_settings_pgsm_max.pl
Updated following expected files for TAP testcase where testcase was updated
with 'where' clause to filter GUC specific row only from settings view.
1) t/expected/001_settings_default.out
2) t/expected/002_settings_pgsm_track.outanning.out
3) t/expected/003_settings_pgms_extract_comments.out
4) t/expected/004_settings_pgsm_track.out
5) t/expected/005_settings_pgsm_enable_query.out
6) t/expected/006_settings_pgsm_overflow_target.out
7) t/expected/007_settings_pgsm_query_shared_buffer.out
8) t/expected/008_settings_pgsm_histogram_buckets.out
9) t/expected/009_settings_pgsm_histogram_max.out
10) t/expected/010_settings_pgsm_histogram_min.out
11) t/expected/011_settings_pgsm_bucket_time.out
12) t/expected/012_settings_pgsm_max_buckets.out
13) t/expected/013_settings_pgsm_normalized_query.out
14) t/expected/014_settings_pgsm_track_utility.out
15) t/expected/015_settings_pgsm_query_max_len.out
16) t/expected/016_settings_pgsm_max.out
17) t/expected/017_execution_stats.out
Updated expected files for following sql based testcases due to changed default
value of pgsm_normalized_query.
1) error_insert_1.out
2) guc_1.out
3) top_query_1.out
Enabled GUC pgsm_normalized_query to 'yes' for following testcases.
1) t/024_check_timings.pl
2) t/025_compare_pgss.pl
3) t/026_shared_blocks.pl
Added the output file for histogram testcase to accomodate behaviour.
Changes suggested by Ibrar to make regression more modular (PG-440), are also
made part of regression so those these can used for future improvements.
Removed commented/unneeded steps from t/017_execution_stats.pl.
Updated testcase tags.sql and output file tags.out to accomodate enabling of
pgsm_extract_comments to yes.
Updated error_1.out output file with required output, it was overwritten in
one of previous commits.
Updated test condition (count of calls and bucket) in t/023_missing_queries.pl
as per comment from Ibrar in PG-461 where PGSM does not track the pg_sleep in
some cases.
This commit brings following changes to this branch:
1) Port changes/additions TAP testing from main branch to this branch, under PG-292.
2) Changes to test cases due to GUCs change, under PG-331.
3) Call counts verfications, under PG-338.
4) Changes to github workflows to accomodate automation for TAP testing, under PG-343.
Changed the default value of pgsm_bucket_time from 300 to 60. Now neither PMM users will need to adjust the default value, nor
restart of PG server will be required for this purpose.
Muhammad Usama
Hamid Akhtar
Naeem Akhter
Ibrar Ahmed
Ibrar Ahmed
Naeem Akhter
Ibrar Ahmed
Diego Fronza
Mikhail Samoylov