* Denormalize prepared statement queries
Added support for extracting query arguments for prepared statements
when `pg_stat_monitor.pgsm_normalized_query` is off.
Previously pg_stat_monitor was unable to extract the arguments for
prepared statements, thus leaving queries with placeholders $1
.. $N instead of the actual arguments.
* Optmize query denormalization
Instead of copying original query text byte by byte, copy data between
query placeholders in chunks, example:
`INSERT INTO foo(a, b, c) VALUES('test', 100, 'test again)'`
Would result in normalized query:
`INSERT INTO foo(a, b, c) VALUES($1, $2, $3)`
The original patch would copy the parts between placeholders byte by
byte, e.g. `INSERT INTO foo(a, b, c) VALUES(`, instead we can copy this
whole block at once, 1 function call and maybe 1 buffer re-allocation
per call.
Also make use of `appendBinaryStringInfo` to avoid calculating string
length as we have this info already.
* Optmize query denormalization(2)
Avoid allocating an array of strings for extracting query argument
values, instead append the current parameter value directly in the
buffer used to store the denormalized query.
This avoids not only unnecessary memory allocations, but also copying
data between temporary memory and the buffer.
* Store denormalized query only under certain constraints
This commit introduces a little optimization along with a feature, it
stores the query in denormalized form only under the circumstances
below:
- The psgm_normalized_query GUC is disabled (off).
- The query is seem for the first time, or the query total
execution time exceeds the mean execution time calculated for
the previous queries.
Having the query which took most execution time along with it's
arguments could help users in further investigating performance issues.
* Fix regression tests
When query normalization is disabled utility queries like SELECT 10+20
are now stored as is, instead of SELECT $1+$2.
Also when functions or sub queries are created the arguments used
internally by the function or subqueries will be replaced by NULL instead
of $1..$N. The actual arguments will be displayed when the function or
subquery is actually invoked.
* Add query denormalization regression test for prepared statements
Ensures that the denormalization of prepared statements is working, also
ensure that a query which takes more time to execute replaces the
previous denormalized query.
* Updated pgsm_query_id regression tests
With the query dernomalization feature, having integer literals used in
sql like 1, or 2 could create some confusion on whether those are
placeholders or constant values, thus this commit updates the
pgsm_query_id regression test to use different integer literals to avoid
confusion.
* Improve query denormalization regression test
Add a new test case:
1. Execute a prepared statement with larger execution time first.
2. Execute the same prepared statement with cheap execution time.
3. Ensures that the denormalized heavy query is not replaced by the
cheaper.
* Format source using pgindent
* Fix top query regression tests on PG 12,13
On PG 12, 13, the internal return instruction in the following function:
```
CREATE OR REPLACE FUNCTION add(int, int) RETURNS INTEGER AS
$$
BEGIN
return (select $1 + $2);
END; $$ language plpgsql;
```
Is stored as SELECT (select expr1 + expr2)
On PG 14 onward it's stored just as SELECT (expr1 + expr2)
* PG-606: New GUC required for enabling/disabling of pgsm_query_id calculation
Adds a new GUC pg_stat_monitor.pgsm_enable_pgsm_query_id to enable/disable
pgsm query id calculation. Apart from that patch also refactors the GUC-related
code to match PostgreSQL conventions.
Moreover, the commit also changes the pgsm_enable_overflow GUC to boolean
instead of enum.
This bug uncovered serious issues with how the data was being stored by PSGM.
So it require a complete redesign.
pg_stat_monitor now stores the data locally within the backend process's local
memory. The data is only stored when the query completes. This reduces the
number of lock acquisitions that were previously needed during various stages
of the execution. Also, this avoids data loss in case the current bucket
changes during execution. Also, the unavailability of jumble state during later
stages of executions was causing pg_stat_monitor to save non-normalized query.
This was a major problem as well.
pg_stat_monitor specific memory context is implemented. It is used for saving
data locally. The context memory callback helps us clear the locally saved data
so that we do not store it multiple times in the shared hash.
As part of this major rewrite, pgss reference in function and variable names
is changed to pgsm. Memory footprint for the entries is reduced, data types
are corrected where needed, and we've removed unused variables, functions and
macros.
This patch was mutually created by:
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
Co-authored-by: Muhammad Usama <muhammad.usama@percona.com>
Regardless of the database or the user, the same query will yield the
same query ID. As part of this, a new column, 'pgsm_query_id', is added.
* pgsm_query_id:
pgsm_query_id has the same data type of int8 as the queryid column. If
the incoming SQL command includes any constants, it internally normalizes
the query to remove those constant values with placeholders. Otherwise,
it uses the query directly to generate the query hash.
Since we no longer depend on the server's parse tree mechanism, we can
generate the same hash for the same query text for all server versions.
Also, it is important to note that the hash being calculated is a database,
schema and user independent. So same query text in different databases
will generate the same hash.
This column is not part of the key; rather, for observability purposes only.
* Regression
SQL test case pgsm_query_id.sql is added to the SQL regression.
Artem Gavrilov
Diego Fronza
Muhammad Usama
Hamid Akhtar