Reimplement the storage mechanism of buckets and query texts
using Dynamic shared memory. Since the dynamic shared memory
can grow into a swap area, so we get the overflow out of the box.
oreover the new design saves the query pointer inside the bucket
and eventually, the query text gets evicted with the bucket recycle.
Finally, the dynamic shared memory hash has a built-in locking
mechanism so we can revisit the whole locking in pg_stat_monitor
has potential for lots of performance improvements
Removing the view for 2.0. Updating the required SQL files to manage
the upgrade. Downgrade from 2.x to 1.x is not supported.
Also part of this fix is the SQL regression. This does not update the
tap test cases.
In line with pg_stat_statments for PG15, This commit adds eight new cumulative
counters for jit operations, making it easier to diagnose how JIT is used in an
installation. And two new columns, temp_blk_read_time, and temp_blk_write_time,
respectively, show the time spent reading and writing temporary file blocks
on disk.
Moreover, The commit also contains a few indentations and API adjustments.
A new column is added to mention that bucket is active or done. there is
some timing based adjustment was required with that too.
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
PG-554: Remove reduntant files and fix regression.
Removed old files with same name and add these files to fix sql regression
on PG 14 & 15.
1- regression/expected/error_1.out
2- regression/expected/error_insert_1.out
3- regression/expected/top_query_1.out
Creating the infrastructure that'll allow using newer versions
of the loadable module with old SQL declarations.
Also updating the build version to 2.0.0-dev
It will be a security problem to provide the internal function access to PUBLIC.
This commit will revoke all permission of internal functions from PUBLIC.
There was a typo while checking the PostgreSQL version in the SQL file. This commit
will fix the typo, and only the necessary columns will be visible in the view.
The query status monitoring code was used to track the current query state, for example,
parsing, executing and finishing. After careful review, we have figured out that
it does not make sense while a lot of time same query is running. Therefore it
is also consuming resources. This commit will remove that feature. The upgrade
SQL from 1.0 - 2.0 is also updated.
Following changes are included in this commit:
1. Updated pgsm.pm to enable runtime loading of PG server version dependent
perl modules that are needed for TAP testing. Similarly removed unneeded
code from this file that is not needed right now.
2. Added generic settings and helper functions to pgsm.pm that could be used
across different test cases.
3.Updated following TAP test case to use pgsm.pm based global settings and helper
functions while making sure that we reduce the clutter and duplicate code in
test cases, where possible.
t/001_settings_default.pl
t/002_settings_pgsm_track_planning.pl
t/003_settings_pgms_extract_comments.pl
t/004_settings_pgsm_track.pl
t/005_settings_pgsm_enable_query_plan.pl
t/006_settings_pgsm_overflow_target.pl
t/007_settings_pgsm_query_shared_buffer.pl
t/008_settings_pgsm_histogram_buckets.pl
t/009_settings_pgsm_histogram_max.pl
t/010_settings_pgsm_histogram_min.pl
t/011_settings_pgsm_bucket_time.pl
t/012_settings_pgsm_max_buckets.pl
t/013_settings_pgsm_normalized_query.pl
t/014_settings_pgsm_track_utility.pl
t/015_settings_pgsm_query_max_len.pl
t/016_settings_pgsm_max.pl
t/017_execution_stats.pl
t/019_insufficient_shared_space.pl
t/020_buffer_overflow.pl
t/021_misc_1.pl
t/022_misc_2.pl
t/023_missing_queries.pl
t/024_check_timings.pl
t/025_compare_pgss.pl
t/026_shared_blocks.pl
t/027_local_blocks.pl
t/028_temp_block.pl
4. Removed following TAP test cases as these are no longer needed and are
covered by existing other test cases.
0001_settings_pgsm_track_planning.pl
0002_settings_pgsm_enable_query_plan.pl
5. Added more out files for histogram sql test cases to cover the behavior for
bucket_start_time and server versions.
regression/expected/histogram_3.out
regression/expected/histogram_4.out
regression/expected/histogram_5.out
regression/expected/histogram_6.out
6. Added following out file that is PG server version 15 specific.
t/expected/007_settings_pgsm_query_shared_buffer.out.15
PG 15 requires additional shared memory and LWLocks requests to be made from the
newly introduced shmem_request_hook and disallows the requests initiated
from outside the hook.
The commit makes moves the additional shared memory and LWLocks requests
from _PG_init to shmem_request_hook for PG15
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Fixing compilation issue, cause by previous commit, where we removed
the benchmarking code. It was causing problem for PostgreSQL-12.
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Remove all benchmarking and debugging code.
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Delete all the SQL.in files because these version-dependent files
are becoming significant in quantity. Now added a single SQL file for which
contains the dynamic SQL based on the PostgreSQL Version.
2 - New SQL files (pg_stat_monitor--2.0.sql) added for pg_stat_monitor version 2.
3 - A new SQL file (pg_stat_monitor--1.0--2.0.sql) is created, which will be
used to upgrade from version 1.0 to 2.0. Currently, this file is empty. But
whenever we add some API changes into 2.0, we need to update that file too.
4 - The control file (pg_stat_monitor.control) is updated for version 2.0.
This change will make the CREATE EXTENSION default to pg_stat_monitor version 2.0
* PG-475: Inconsistent behaviour of PGSM
Reverting the bucket locking mechanism to previous behavior. This has
a lot of room for improvement that needs to be part of a major refactoring
in the 2.x release.
* PG-481 Release notes 1.1.0 (#294)
modified: RELEASE_NOTES.md
* PG-500: Bump the version of pg_stat_monitor to 1.1.0 (#297)
* PG-501: Missing Buckets and incorrect calls count. (#298)
prev_bucket_sec holds the actual time at which the previous bucket was created
and it is used to compute if the previous bucket time has elapsed and when is
the time to create a new one. But since the bucket start time is rounded down
to logical time window start, that makes the prev_bucket_sec and bucket start
time out of sync with each other, and depending on the query arrival time there
is a high probability that a bucket gets missed especially when the last bucket
was created around the end of the bucket time window.
Solution is to keep the prev_bucket_sec and bucket start time in-sync.
Moreover, we are using the unint64 for storing the prev_bucket_sec which is kind
of an overkill and a simple uint should be good enough for the purpose. But that
change can be taken up as part of the create-bucket function refactoring task.
* PG-501: Missing Buckets and incorrect calls count.
Ensuring the outer bound for the bucket is an exclusive boundary and it
as it belongs to the next bucket. To explain the point further, a set of
five second bucket would be:
Bucket 1: 00:00:00.00 -> 00:00:04.99...
Bucket 2: 00:00:00.05 -> 00:00:09.99...
Bucket 3: 00:00:00.10 -> 00:00:14.99...
...
Co-authored-by: Ibrar Ahmed <ibrar.ahmed@percona.com>
Co-authored-by: Anastasia Alexandrova <anastasia.alexandrova@percona.com>
Co-authored-by: Muhammad Usama <m.usama@gmail.com>
As per discussion in the team meeting, removed status badges for Pg-11~13 jobs
from README. We should only be displaying the status badges for the latest PG
version (GA) and code coverage.
Perl tap tests dependencies were missing, those are being installed now.
Furthermore, workflow required some changes to enable logs to be available in
in case of a tap tests failure(s).
Ibrar Ahmed
Muhammad Usama
Naeem Akhter
Hamid Akhtar
Naeem Akhter
Kai Wagner
Puneet Kala
Hamid Akhtar
Vadim Yalovets