Updated sql files (pg_stat_monitor_settings view).
Using right variable name and level checking on pgss_store:
key.toplevel = ((exec_nested_level + plan_nested_level) == 0);
The issue with wrong query call count was taking place during transition
to a new bucket, the process is shortly describe bellow:
1. Scan for pending queries in previous bucket.
2. Add pending queries to the new bucket id.
3. Remove pending queries from previous bucket id.
The problem is that when switching to a new bucket, we reset query
statistics for a given entry being processed, so, for example, if the
pending query had a call count of 10 (9 of which were finished, 10th is
the pending one), if we move this query to the new bucket, the entry
will have its stats reseted, clearing the query call count to zero.
To solve the problem, whenever a pending query is detected, if the entry
has a call count > 1, we mark it as finished, and don't remove it from
the previous bucket in order to keep its statistics, then we move just
the pending query (10th in the example) to the new bucket id.
Another issue is that when moving a entry to a new bucket, we missed
copying the query position from the previous entry, which is used to
locate the query text in the query buffer:
hash_entry_dealloc():291
new_entry->query_pos = old_entry->query_pos;
The issue is that between changing GUC "track" from track='top' to
track='all' the queries are executing using previous state of
track='top', to fix that we sleep 1 second after calling
pg_reload_conf() to ensure that queries will run with new settings.
cmd_type: Added missing DROP TABLE t2;
guc: Adjusted to match the updated settings view, which now display
boolean values as 'yes' and 'no', also added the 'options' column
to the output.
guc_1: Handle PostgreSQL versions <= 12 which don't have the
track_planning feature.
rows.out: Added missing DROP TABLE t2. Also removed the line 'ERROR:
relation "t2" already exists' since we fixed the problem in
cmd_type regression.
top_query: Handling both track = 'top' and track = 'all' cases.
top_query_1: On PostgreSQL >= 14 the sub query from the procedure is
stored as (select $1 + $2), whereas on PG <= 13 it is
stored as SELECT (select $1 + $2).
This new GUC allows users to select which statements are tracked by
pg_stat_monitor:
- 'top': Default, track only top level queries.
- 'all': Track top along with sub/nested queries.
- 'none': Disable query monitoring.
To avoid redudancy, now users disable pg_stat_monitor by setting
pg_stat_monitor.track = 'none', similar to pg_stat_statements.
This new GUC is an enumeration, so the pg_stat_monitor_settings view was
adjusted to add a new column 'options' which lists the possible values
for the field.
The "value" and "default_value" columns in the pg_stat_monitor_settings
view was adjusted to be of type text, so we can better display the
enumeration values. Also the boolean types now have their values
displayed as either 'yes' or 'no' to easily distinguish them from the
integer types.
Can't call elog() function from inside the pgsm_log as the pgss_hash
lock could be already acquired in exclusive mode, since elog() triggers
the psmg_emit_log hook, when it calls pgss_store it will try to acquire
the pgss_hash lock again, leading to a deadlock.
As the query normalization and query cleaning is always done in the
right place (pgss_store), no more parsed queries have a trailling comma
';' at the end.
Also, on error regression test, after fixing some problems with utility
related queries, we now have two entries for the RAISE WARNING case, the
first entry is the utility query itself, the second entry is the error
message logged by emit_log_hook.
Some queries have the order adjusted due to the fix introduced by the
previous commits.
This commit introduces serveral improvements:
1. Removal of pgss_store_query and pgss_store_utility functions: To
store a query, we just use pgss_store(), this makes the code more
uniform.
2. Always pass the query length to the pgss_store function using parse
state from PostgreSQL to avoid calculating query length again.
3. Always clean the query (extra spaces, update query location) in
pgss_store.
4. Normalize queries right before adding them to the query buffer, but
only if user asked for query normalization.
5. Correctly handle utility queries among different PostgreSQL versions:
- A word about how utility functions are handled on PG 13 and later
versions:
- On PostgreSQL <= 13, we have to compute a query ID, on later
versions we can call EnableQueryId() to inform Postmaster we
want to enable query ID computation.
- On PostgreSQL <= 13, post_parse hook is called after process
utility hook, on PostgreSQL >= 14, post_parse hook is called
before process utility functions.
- Based on that information, on PostgreSQL <= 13 / process utility,
we pass 0 as queryid to the pgss_store function, then we calculate a
queryid after cleaning the query (CleanQueryText) using
pgss_hash_string.
- On PostgreSQL 14 onward, post_parse() is called before
pgss_ProcessUtility, we Clear queryId for prepared statements
related utility, on process utility hook, we save the query ID for
passing it to the pgss_store function, but mark the query ID with
zero to avoid instrumenting it again on executor hooks.
After couple CPU profiling sessions with perf, it was detected that the
function pgstat_fetch_stat_numbackends() is very expensive, reading the
implementation on PostgreSQL's backend_status.c just confirmed that.
We use that function on pg_stat_monitor to retrieve the application name
and IP address of the client, we now cache the results in order to avoid
calling it for every query being processed.
If pgsm_overflow_target is ON (default, 1) and the query buffer
overflows, we now dump the buffer and keep track of how many times
pg_stat_monitor changed bucket since that.
If an overflow happen again before pg_stat_monitor cycle through
pgsm_max_buckets buckets (default 10), then we don't dump the buffer
again, but instead report an error, this ensures that only one dump file
of size pgsm_query_shared_buffer will be in disk at any time, avoiding
slowing down queries to the pg_stat_monitor view.
As soon as pg_stat_monitor cycles through all buckets, we remove the
dump file and reset the counter (pgss->n_bucket_cycles).
pgss_ExecutorEnd: Avoid unnecessary memset(plan_info, 0, ...).
We only use this object if the condition below is true, in which case we
already initialize all the fields in the object, also we now store the
plan string length (plan_info.plan_len) to avoid calling strlen on it
again later:
if (queryDesc->operation == CMD_SELECT && PGSM_QUERY_PLAN) {
... here we initialize plan_info
If the condition is false, then we pass a NULL PlanInfo* to the
pgss_store to avoid more unnecessary processing.
pgss_planner_hook: Similar, avoid memset(plan_info, 0, ...) this object
is not used here, so we pass NULL to pgss_store.
pg_get_application_name: Remove call to strlen, snprintf already give us
the calculated string length, so we just return it.
pg_get_client_addr: Cache localhost, avoid calling
ntohl(inet_addr("127.0.0.1")) all the time.
pgss_update_entry: Make use of PlanInfo->plan_len, avoiding a call to
strlen again.
intarray_get_datum: Init the string by setting the first byte to '\0'.
The memory area reserved for query text (pgsm_query_shared_buffer) was
divided evenly for each bucket, this allowed to have the same query,
e.g. "SELECT 1", duplicated in different buckets, thus wasting space.
This commit fix the query text duplication by adding a new hash table
whose only purpose is to verify if a given query is already added to the
buffer (by using the queryID).
This allows different buckets that share the same query to point to a
unique entry in the query buffer (pgss_qbuf).
When pg_stat_monitor moves to a new bucket id, by avoiding adding a
query that already exists in the buffer it can also save some CPU time.
Before PostgreSQL 14, HASH_STRINGS flag was not available when creating
a hash table with ShmemInitHash().
Use HASH_BLOBS for previous PostgreSQL versions.
The pg_stat_monitor_errors view was created in order to help inspecting
internal errors that may occur in pg_stat_monitor module itself, it
contains the error message, its severity, the last time the error occurred
and the number of times that any given error ocurred.
Implementation details:
- A new lwlock was added to the pgssSharedState structure in order to
control access to the errors hash table.
- Increased RequestNamedLWLockTranche() no. of locks requested to 2.
- The function GetNamedLWLockTranche() returns an array of locks, we
use the first lock for controlling access to the usual buckets hash
table, and the second one to control access to the errors hash table.
- During module initialization (_PG_init) we increased the amount of
shared memory space requested to include space to the new errors hash
table: RequestAddinShmemSpace(... + pgsm_errors_size())
- The implementation in pgsm_errors.c simple uses a hash table to track
error messages, the message is also used as the key.
There were couple issues to handle, the main one was that our log hook
(pgsm_emit_log_hook) was being called after the shared memory hook
completed (pgss_shmem_startup) but before PostgreSQL boostraping code
finished, thus triggering the following assertion during a call to
LWLockAcquire():
Assert(!(proc == NULL && IsUnderPostmaster));
proc is a pointer to MyProc, a PostgreSQL's shared global variable that
was not yet initalized by PostgreSQL.
We must also check for a NULL pointer return in pg_get_backend_status()
the pgstat_fetch_stat_local_beentry() function may return a NULL pointer
during initialization, in which case we use "127.0.0.1" for the client
address, and "postmaster" for application name.
Diego Fronza
Ibrar Ahmed
Umair Shahid
Vadim Yalovets
EvgeniyPatlan
Oleksandr Miroshnychenko
Anastasia Alexadrova