Following changes are included in this commit:
1. Updated pgsm.pm to enable runtime loading of PG server version dependent
perl modules that are needed for TAP testing. Similarly removed unneeded
code from this file that is not needed right now.
2. Added generic settings and helper functions to pgsm.pm that could be used
across different test cases.
3.Updated following TAP test case to use pgsm.pm based global settings and helper
functions while making sure that we reduce the clutter and duplicate code in
test cases, where possible.
t/001_settings_default.pl
t/002_settings_pgsm_track_planning.pl
t/003_settings_pgms_extract_comments.pl
t/004_settings_pgsm_track.pl
t/005_settings_pgsm_enable_query_plan.pl
t/006_settings_pgsm_overflow_target.pl
t/007_settings_pgsm_query_shared_buffer.pl
t/008_settings_pgsm_histogram_buckets.pl
t/009_settings_pgsm_histogram_max.pl
t/010_settings_pgsm_histogram_min.pl
t/011_settings_pgsm_bucket_time.pl
t/012_settings_pgsm_max_buckets.pl
t/013_settings_pgsm_normalized_query.pl
t/014_settings_pgsm_track_utility.pl
t/015_settings_pgsm_query_max_len.pl
t/016_settings_pgsm_max.pl
t/017_execution_stats.pl
t/019_insufficient_shared_space.pl
t/020_buffer_overflow.pl
t/021_misc_1.pl
t/022_misc_2.pl
t/023_missing_queries.pl
t/024_check_timings.pl
t/025_compare_pgss.pl
t/026_shared_blocks.pl
t/027_local_blocks.pl
t/028_temp_block.pl
4. Removed following TAP test cases as these are no longer needed and are
covered by existing other test cases.
0001_settings_pgsm_track_planning.pl
0002_settings_pgsm_enable_query_plan.pl
5. Added more out files for histogram sql test cases to cover the behavior for
bucket_start_time and server versions.
regression/expected/histogram_3.out
regression/expected/histogram_4.out
regression/expected/histogram_5.out
regression/expected/histogram_6.out
6. Added following out file that is PG server version 15 specific.
t/expected/007_settings_pgsm_query_shared_buffer.out.15
PG 15 requires additional shared memory and LWLocks requests to be made from the
newly introduced shmem_request_hook and disallows the requests initiated
from outside the hook.
The commit makes moves the additional shared memory and LWLocks requests
from _PG_init to shmem_request_hook for PG15
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Fixing compilation issue, cause by previous commit, where we removed
the benchmarking code. It was causing problem for PostgreSQL-12.
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Remove all benchmarking and debugging code.
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Delete all the SQL.in files because these version-dependent files
are becoming significant in quantity. Now added a single SQL file for which
contains the dynamic SQL based on the PostgreSQL Version.
2 - New SQL files (pg_stat_monitor--2.0.sql) added for pg_stat_monitor version 2.
3 - A new SQL file (pg_stat_monitor--1.0--2.0.sql) is created, which will be
used to upgrade from version 1.0 to 2.0. Currently, this file is empty. But
whenever we add some API changes into 2.0, we need to update that file too.
4 - The control file (pg_stat_monitor.control) is updated for version 2.0.
This change will make the CREATE EXTENSION default to pg_stat_monitor version 2.0
* PG-475: Inconsistent behaviour of PGSM
Reverting the bucket locking mechanism to previous behavior. This has
a lot of room for improvement that needs to be part of a major refactoring
in the 2.x release.
* PG-481 Release notes 1.1.0 (#294)
modified: RELEASE_NOTES.md
* PG-500: Bump the version of pg_stat_monitor to 1.1.0 (#297)
* PG-501: Missing Buckets and incorrect calls count. (#298)
prev_bucket_sec holds the actual time at which the previous bucket was created
and it is used to compute if the previous bucket time has elapsed and when is
the time to create a new one. But since the bucket start time is rounded down
to logical time window start, that makes the prev_bucket_sec and bucket start
time out of sync with each other, and depending on the query arrival time there
is a high probability that a bucket gets missed especially when the last bucket
was created around the end of the bucket time window.
Solution is to keep the prev_bucket_sec and bucket start time in-sync.
Moreover, we are using the unint64 for storing the prev_bucket_sec which is kind
of an overkill and a simple uint should be good enough for the purpose. But that
change can be taken up as part of the create-bucket function refactoring task.
* PG-501: Missing Buckets and incorrect calls count.
Ensuring the outer bound for the bucket is an exclusive boundary and it
as it belongs to the next bucket. To explain the point further, a set of
five second bucket would be:
Bucket 1: 00:00:00.00 -> 00:00:04.99...
Bucket 2: 00:00:00.05 -> 00:00:09.99...
Bucket 3: 00:00:00.10 -> 00:00:14.99...
...
Co-authored-by: Ibrar Ahmed <ibrar.ahmed@percona.com>
Co-authored-by: Anastasia Alexandrova <anastasia.alexandrova@percona.com>
Co-authored-by: Muhammad Usama <m.usama@gmail.com>
As per discussion in the team meeting, removed status badges for Pg-11~13 jobs
from README. We should only be displaying the status badges for the latest PG
version (GA) and code coverage.
Perl tap tests dependencies were missing, those are being installed now.
Furthermore, workflow required some changes to enable logs to be available in
in case of a tap tests failure(s).
With the documentation moved to a separate repo, remove the docs from this repository to avoid confusion where to contribute to docs. Also, removing a GH action and unpublishing the GH pages site with the docs being built on Render. Updated README and Contributing guide with the new docs repo.
deleted: .github/workflows/doc-build.yml
modified: README.md
deleted: docs/COMPARISON.md
deleted: docs/REFERENCE.md
deleted: docs/USER_GUIDE.md
deleted: docs/_images/percona-favicon.ico
deleted: docs/_images/percona-logo.svg
deleted: docs/css/percona.css
deleted: docs/css/toctree.css
deleted: docs/index.md
deleted: docs/js/version-select.js
deleted: docs/overrides/404.html
deleted: docs/setup.md
deleted: requirements-doc.txt
Adding SQL file for PG15. Initial version is a replica of PG14 SQL file.
Changes will be required in this file to fully support newly introduced columns
in PG15.
This currently fixes compilation and make install targets.
Also, updated the first line comments where the SQL version was mentioend as
1.1. The version remains at 1.0 for the time being.
The Makefile is refactored and unnecessary conditional statements are removed
in favor of handling version separately.
Also, the clean target is updated to remove generated files including results
folder and the ${DATA} file.
This commit does not make any changes to the makefile. It simply ensures that
the test cases work with PG12. Necessary skip statements are added. Makefile
changes are to be done as part refactoring the Makefile.
Updated status badges links in README that were not carrying the correct links, workflows
(Actions) for older links have been removed from repo as those have been
replaced by distribution specific (PGDG and PPG) workflows.
1- Removed unsupported options from configure.
2- Reformatted files to break lines at 80 charachters.
3- Moved multiple 'apt install' and rm commands into a single one.
4- All workflows use and install llvm + clang version 11.
5- Updated workflows to ubuntu 22.04 where we are building PG server.
6- Added .vscode folder/files to .gitignore file.
1) Updated code coverage workflow with required changes so that we point to
Codecov onwards for coverage stats.
2) Updated README.md file so that badge for Code Coverage, points to CodeCov
coverage stats for PGSM.
3) Added .DS_STORE to .gitignore file.
Regression was failing in main branch after merging REL_1_STABLE. Changes to
stablise the regression are summarised below.
Updated following TAP testcase with 'where' clause for filtering GUC specific row
from settings view. Intention is to fetch only GUC value and test it.
1) t/001_settings_default.pl
2) t/002_settings_pgsm_track_planning.pl
3) t/003_settings_pgms_extract_comments.pl
4) t/004_settings_pgsm_track.pl
5) t/005_settings_pgsm_enable_query_plan.pl
6) t/006_settings_pgsm_overflow_target.pl
7) t/007_settings_pgsm_query_shared_buffer.pl
8) t/008_settings_pgsm_histogram_buckets.pl
9) t/009_settings_pgsm_histogram_max.pl
10) t/010_settings_pgsm_histogram_min.pl
11) t/011_settings_pgsm_bucket_time.pl
12) t/012_settings_pgsm_max_buckets.pl
13) t/013_settings_pgsm_normalized_query.pl
14) t/014_settings_pgsm_track_utility.pl
15) t/015_settings_pgsm_query_max_len.pl
16) t/016_settings_pgsm_max.pl
Updated following expected files for TAP testcase where testcase was updated
with 'where' clause to filter GUC specific row only from settings view.
1) t/expected/001_settings_default.out
2) t/expected/002_settings_pgsm_track.outanning.out
3) t/expected/003_settings_pgms_extract_comments.out
4) t/expected/004_settings_pgsm_track.out
5) t/expected/005_settings_pgsm_enable_query.out
6) t/expected/006_settings_pgsm_overflow_target.out
7) t/expected/007_settings_pgsm_query_shared_buffer.out
8) t/expected/008_settings_pgsm_histogram_buckets.out
9) t/expected/009_settings_pgsm_histogram_max.out
10) t/expected/010_settings_pgsm_histogram_min.out
11) t/expected/011_settings_pgsm_bucket_time.out
12) t/expected/012_settings_pgsm_max_buckets.out
13) t/expected/013_settings_pgsm_normalized_query.out
14) t/expected/014_settings_pgsm_track_utility.out
15) t/expected/015_settings_pgsm_query_max_len.out
16) t/expected/016_settings_pgsm_max.out
17) t/expected/017_execution_stats.out
Updated expected files for following sql based testcases due to changed default
value of pgsm_normalized_query.
1) error_insert_1.out
2) guc_1.out
3) top_query_1.out
Enabled GUC pgsm_normalized_query to 'yes' for following testcases.
1) t/024_check_timings.pl
2) t/025_compare_pgss.pl
3) t/026_shared_blocks.pl
Added the output file for histogram testcase to accomodate behaviour.
Changes suggested by Ibrar to make regression more modular (PG-440), are also
made part of regression so those these can used for future improvements.
Removed commented/unneeded steps from t/017_execution_stats.pl.
Updated testcase tags.sql and output file tags.out to accomodate enabling of
pgsm_extract_comments to yes.
Updated error_1.out output file with required output, it was overwritten in
one of previous commits.
Updated test condition (count of calls and bucket) in t/023_missing_queries.pl
as per comment from Ibrar in PG-461 where PGSM does not track the pg_sleep in
some cases.
PG-456: Running pgindent to make source indentation/spacing PostgreSQLCompatible.
PostgreSQL uses pgindent from time to time to make source code PostgreSQL
style guide compatible, it is a very long time since we have not done that.
Commit fixes a lot of indentation and spacing issues.
Co-authored-by: Hamid Akhtar <hamid.akhtar@gmail.com>
PG-453: Normalize query does not work with INSERT statements.
The commit fixes the issue similar to pg_stat_statements. Jumble query skips in
case of INSERT statements to avoid the duplicate queryid, but in another
commit we already solved that problem with another way.
The buckets are now created with the start time a modulus of the bucket time size.
So if we have a 10 second bucket, the start times would reflect that:
- Bucket1: 00:00:00
- Bucket2: 00:00:10
- Bucket3: 00:00:20
...
Previously, the start time of the bucket was aligned with the first query that
arrives in that bucket. However, now the behaviour is changed. So, even if the
first query for bucket 2 arrives at 00:00:13, the start time would still be set
to 00:00:10.
This change now makes the bucketing separated out by fixed time windows so that
external applications can easily consume that data and chart it.
Also, as part of this change, locking of pgss is updated now and extended
to last the bucket related changes.
Kai Wagner
Puneet Kala
Naeem Akhter
Muhammad Usama
Ibrar Ahmed
Hamid Akhtar
Hamid Akhtar
Vadim Yalovets
Naeem Akhter
Anastasia Alexadrova
Lenz Grimmer