external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,866 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

4562 Commits (4c9863c0fbc98864656f3eee543212197aff5eb1)

Author SHA1 Message Date
gindibay 3a0a5ae344 Fixes missin ok issue 2023-10-24 15:13:22 +03:00
gindibay b5cbc048b7 Fixes missing ok issue 2023-10-24 15:12:08 +03:00
gindibay f779947b2b Removes unnecessary logs 2023-10-24 14:57:36 +03:00
gindibay 1dcee370e1 Fixes create_drop_database_test 2023-10-24 14:44:34 +03:00
gindibay fb9f75bb24 Removes unnecessary logs 2023-10-24 14:43:12 +03:00
gindibay e9e64a69c1 Fixes indentation 2023-10-24 14:18:10 +03:00
gindibay 73f0db2aed Fixes create and drop database transaction use 2023-10-24 14:09:36 +03:00
gindibay c9dae2684f tests db as role 2023-10-24 10:09:13 +03:00
Naisila Puka 10198b18e8
Technical readme small fixes (#7261) 2023-10-23 13:43:43 +03:00
Naisila Puka 1fe16fa746
Remove unnecessary pre-fastpath code (#7262) 
This code was here because we first implemented
`fast path planner` via
[#2606](https://github.com/citusdata/citus/pull/2606)
and then later `deferred pruning`
[#3369](https://github.com/citusdata/citus/pull/3369)
So, for some years, this code was useful.
 2023-10-23 13:01:48 +03:00
gindibay 690276c516 Fixes indentation 2023-10-23 12:35:53 +03:00
gindibay ea0a908702 Fixes pg_dist_objects record in coordinators 2023-10-23 12:31:25 +03:00
gindibay dad81042a5 Adds undistribute sql 2023-10-23 10:01:18 +03:00
zhjwpku 2d1444188c
Fix wrong comments around HasDistributionKey() (#7223) 
HasDistributionKey & HasDistributionKeyCacheEntry returns true when the
corresponding table has a distribution key, the comments state the
opposite,
which should be fixed.

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
 2023-10-18 10:53:00 +02:00
Onur Tirtir db13afaa7b
Fix flaky columnar_create.sql test (#7266) 2023-10-17 16:58:17 +03:00
Gürkan İndibay 71a4633dad
Fixes typo and renames multi_process_utility (#7259) 2023-10-17 16:39:37 +03:00
Jelte Fennema-Nio 788e09a39a
Add a test for citus_shards where table names have spaces (#7224) 
There was a bug reported for previous versions of Citus where
shard\_size was returning NULL for tables with spaces in them. It works
fine on the main branch though, but I'm still adding a test for this to
the main branch because it seems a good test to have.
 2023-10-16 11:38:24 +02:00
gindibay 1e91b144e2 Fixed review issues 2023-10-14 20:49:26 +03:00
gindibay a497a78239 Fixes review comments 2023-10-14 20:39:39 +03:00
gindibay c6d1ef9e41 Rollbacks enable_create_database_propagation 2023-10-14 20:19:18 +03:00
gindibay 674fd3226c Sets enable_create_database_propagation true 2023-10-14 19:50:32 +03:00
gindibay 3a6fdada11 Changes if to switch statements 2023-10-14 05:35:34 +03:00
gindibay ca5e234821 Adds EnableCreateDatabasePropagation for drop db 2023-10-13 14:17:14 +03:00
gindibay abf9dd9e59 Adds EnableCreateDatabasePropagation check 2023-10-13 14:17:14 +03:00
gindibay 333f77a267 Fixes distributed_object management 2023-10-13 14:17:14 +03:00
gindibay c31479d2eb Fixes review notes 2023-10-13 14:17:14 +03:00
gindibay a274ffcb51 Adds If exists statement for drop database 2023-10-13 14:17:14 +03:00
gindibay 8cc67921cb Fixes review comments 2023-10-13 14:17:14 +03:00
gindibay 875f347238 Fixes indentation 2023-10-13 14:17:14 +03:00
gindibay f2092d2f44 Clarifies the code 2023-10-13 14:17:14 +03:00
gindibay 7bdeb2f866 Fixes indentation 2023-10-13 14:17:14 +03:00
gindibay fb6dd413ad Fixes PostprocessCreateDatabaseStmt comment 2023-10-13 14:17:14 +03:00
gindibay d88a1e7785 Fixes compile error 2023-10-13 14:17:14 +03:00
gindibay d0ef13dc89 Fixes review comments 2023-10-13 14:17:14 +03:00
gindibay 480c22acc2 Fixes the comment 2023-10-13 14:17:14 +03:00
gindibay d778bb1ed6 Adds PG_VERSION_15 constant 2023-10-13 14:17:14 +03:00
gindibay d46a8394a3 Fixes compile error 2023-10-13 14:17:14 +03:00
gindibay df715a0402 Fixes indent 2023-10-13 14:17:14 +03:00
gindibay 134ec8f428 Fixes compile error for PG 14 2023-10-13 14:17:14 +03:00
gindibay 6096a0ed57 Adds missing types 2023-10-13 14:17:14 +03:00
gindibay e7f8058016 Removes T_Boolean since it breaks pg14 compile 2023-10-13 14:17:14 +03:00
gindibay 82358fb37d Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 0bf1c2c139 Refactors the code to make more managable 2023-10-13 14:17:13 +03:00
gindibay 09d201d1a8 Fixes pg15 output 2023-10-13 14:17:13 +03:00
gindibay 7bccf93183 Fixes icu locale 2023-10-13 14:17:13 +03:00
gindibay d55ecc23ae Updates test results 2023-10-13 14:17:13 +03:00
gindibay 32d74d9521 Fixes collate and ctype 2023-10-13 14:17:13 +03:00
gindibay ddea1c897b Removes iculocale 2023-10-13 14:17:13 +03:00
gindibay da46878a8f Fixes Locale 2023-10-13 14:17:13 +03:00
gindibay 031b6480f2 Fixes locale 2023-10-13 14:17:13 +03:00
gindibay 576264ceca Fixes UTF error 2023-10-13 14:17:13 +03:00
gindibay 3050b0eab9 Fixes pg15 tests 2023-10-13 14:17:13 +03:00
gindibay 214ebc4374 Fixes encodings for pg16 2023-10-13 14:17:13 +03:00
gindibay 6f36556724 Revert "Removes pg15 changes to test unusal failures" 
This reverts commit 85963e9cb5.
 2023-10-13 14:17:13 +03:00
gindibay a7112fb965 Adds tablespace to test tablespace statement 2023-10-13 14:17:13 +03:00
gindibay 6913496f42 Fixes tests 2023-10-13 14:17:13 +03:00
gindibay 94abb07409 Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 1f578791bb Fixes pg_dist_object delete problem 2023-10-13 14:17:13 +03:00
gindibay 51d1560b34 Adds OID to see the error cause 2023-10-13 14:17:13 +03:00
gindibay c3d773ac6f Changes test order 2023-10-13 14:17:13 +03:00
gindibay 1d9c02fab9 Changes schedule location 2023-10-13 14:17:13 +03:00
gindibay 5eb87839b6 Rollbacks last test 2023-10-13 14:17:13 +03:00
gindibay acece9818c Rollbacks pg_regress_multi 2023-10-13 14:17:13 +03:00
gindibay c7304abd54 Fixes pg15 file 2023-10-13 14:17:13 +03:00
gindibay 14618878c8 Tests error cause 2023-10-13 14:17:13 +03:00
gindibay 29093e042a Removes pg15 changes to test unusal failures 2023-10-13 14:17:13 +03:00
gindibay aa47ca8bda Fixes tablespace results 2023-10-13 14:17:13 +03:00
gindibay 4c79788936 Fixes tablespace error 2023-10-13 14:17:13 +03:00
gindibay b01c9ffb45 Removes unnecessary fileds from selects 2023-10-13 14:17:13 +03:00
gindibay d0bd20c9b1 Fixes tests 2023-10-13 14:17:13 +03:00
gindibay 25488afccd Changes encoding 2023-10-13 14:17:13 +03:00
gindibay a5e0b9411a Adds tests for create and drop database 2023-10-13 14:17:13 +03:00
gindibay 8c98369e21 Fixes affected tests 2023-10-13 14:17:13 +03:00
gindibay 5c50d58757 Changes EnableCreateDatabasePropagation to false 
to fix failing tests
 2023-10-13 14:17:13 +03:00
gindibay 3735bef156 Fixes sql comments 2023-10-13 14:17:13 +03:00
gindibay cef9ac702e Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 1e7c18cb61 Removes unnecessary code 2023-10-13 14:17:13 +03:00
gindibay 40893e6ffc Removes unnecessary files 2023-10-13 14:17:13 +03:00
gindibay eb1f093381 Fixes runtime and compile errors 2023-10-13 14:17:13 +03:00
gindibay ca36833b5c Fix for Pg16 compile error 2023-10-13 14:16:43 +03:00
gindibay bd398a627b Fixes compile errors 2023-10-13 14:16:43 +03:00
gindibay 39cb7977ce Fixes additional compile errors 2023-10-13 14:16:43 +03:00
gindibay 0d5c2f9350 Adds first commit 2023-10-13 14:16:43 +03:00
Emel Şimşek e9035f6d32
Send keepalive messages in split decoder periodically to avoid wal receiver timeouts during large shard splits. (#7229) 
DESCRIPTION: Send keepalive messages during the logical replication
phase of large shard splits to avoid timeouts.

During the logical replication part of the shard split process, split
decoder filters out the wal records produced by the initial copy. If the
number of wal records is big, then split decoder ends up processing for
a long time before sending out any wal records through pgoutput. Hence
the wal receiver may time out and restarts repeatedly causing our split
driver code catch up logic to fail.

Notes: 

1. If the wal_receiver_timeout is set to a very small number e.g. 600ms,
it may time out before receiving the keepalives. My tests show that this
code works best when the` wal_receiver_timeout `is set to 1minute, which
is the default value.

2. Once a logical replication worker time outs, a new one gets launched.
The new logical replication worker sets the pg_stat_subscription columns
to initial values. E.g. the latest_end_lsn is set to 0. Our driver logic
in `WaitForGroupedLogicalRepTargetsToCatchUp` can not handle LSN value
to go back. This is the main reason for it to get stuck in the infinite
loop.
 2023-10-09 22:33:08 +03:00
Nils Dijk 6d8725efb0
Fix leaking of memory and memory contexts in Foreign Constraint Graphs (#7236) 
DESCRIPTION: Fix leaking of memory and memory contexts in Foreign
Constraint Graphs

Previously, every time we (re)created the Foreign Constraint
Relationship Graph, we created a new Memory Context while loosing a
reference to the previous context. This old context could still have
left over memory in there causing a memory leak.

With this patch we statically have one memory context that we lazily
initialize the first time we create our foreign constraint relationship
graph. On every subsequent creation, beside destroying our previous
hashmap we also reset our memory context to remove any left over
references.
 2023-10-09 13:05:51 +02:00
Onur Tirtir 858d99be33
Take improvement_threshold into the account in citus_add_rebalance_strategy() (#7247) 
DESCRIPTION: Makes sure to take improvement_threshold into the account
in `citus_add_rebalance_strategy()`.

Fixes https://github.com/citusdata/citus/issues/7188.
 2023-10-09 13:13:08 +03:00
Önder Kalacı 7d6c401dd3
Update technical readme (#7248) 
Fix a wrong query, reported by @naisila
 2023-10-06 13:37:37 +03:00
Önder Kalacı 0dca65c84d
Addd missing image to Technical Readme (#7243) 
DESCRIPTION: PR description that will go into the change log, up to 78
characters
 2023-09-29 22:24:10 +02:00
Önder Kalacı 185ac5e01e
Citus Technical Readme (#7207) 
This commit aims to add a comprehensive guide that covers all essential
aspects of Citus, including planning, execution, locking mechanisms,
shard moves, 2PC, and many other major components of Citus.

Co-authored-by: Marco Slot <marco.slot@gmail.com>
 2023-09-29 16:50:52 +03:00
dependabot[bot] c323f49e83
Bump cryptography from 41.0.3 to 41.0.4 in /src/test/regress (#7231) 
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.3
to 41.0.4.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nils Dijk <nils@citusdata.com>
 2023-09-27 15:36:58 +02:00
Onur Tirtir 27ac44eb2a
Fix mixed Citus upgrade tests (#7218) 
When testing rolling Citus upgrades, coordinator should not be upgraded
until we upgrade all the workers.

---------

Co-authored-by: Jelte Fennema-Nio <github-tech@jeltef.nl>
 2023-09-26 17:52:52 +03:00
Nils Dijk b87fbcbf79
Shard moves/isolate report LSN's in lsn format (#7227) 
DESCRIPTION: Shard moves/isolate report LSN's in lsn format

While investigating an issue with our catchup mechanism on certain
postgres versions we noticed we print LSN's in the format of the native
long type. This is an uncommon representation for LSN's in postgres
logs.

This patch changes the output of our log message to go from the long
type representation to the native LSN type representation. Making it
easier for postgres users to recognize and compare LSN's with other
related reports.

example of new output:
```
2023-09-25 17:28:47.544 CEST [11345] LOG:  The LSN of the target subscriptions on node localhost:9701 have increased from 0/0 to 0/E1ED20F8 at 2023-09-25 17:28:47.544165+02 where the source LSN is 1/415DCAD0
```
 2023-09-26 13:47:50 +02:00
Gürkan İndibay 7fa109c977
Adds alter user missing features (#7204) 
DESCRIPTION: Adds alter user rename propagation and enriches alter user
tests

---------

Co-authored-by: Jelte Fennema <github-tech@jeltef.nl>
 2023-09-26 12:28:07 +03:00
Onur Tirtir 111b4c19bc
Make sure to disallow creating a replicated distributed table concurrently (#7219) 
See explanation in https://github.com/citusdata/citus/issues/7216.
Fixes https://github.com/citusdata/citus/issues/7216.

DESCRIPTION: Makes sure to disallow creating a replicated distributed
table concurrently
 2023-09-25 11:14:35 +03:00
Nils Dijk 0f28a69f12
Use the $(DLSUFFIX) instead of hard coded extensions for cdc (#7221) 
When cdc got added the makefiles hardcoded the `.so` extension instead
of using the platform specifc `$(DLSUFFIX)` variable used by `pgxs.mk`.
Also don't remove installed cdc artifacts on `make clean`.
 2023-09-22 16:24:18 +02:00
Jelte Fennema-Nio 71e556e090
Remove useless test output (#7209) 
This was sometimes failing when running locally due to some local shard
still existing due to. This fixes that. We normally silence all
`drop schema cascade` output like this anyway to avoid unnecessary
diffs when modifying a test later on.
 2023-09-19 14:12:46 +02:00
Naisila Puka 4e46708789
Adds PostgreSQL 16.0 Support (#7201) 
This commit concludes PG16.0 Support in Citus.

The main PG16 support work has been done for 16beta3
https://github.com/citusdata/citus/pull/6952
There was some extra work needed for 16rc1
https://github.com/citusdata/citus/pull/7173
And this PR yet introduces some extra work needed to 16.0 :)

`pgstat_fetch_stat_local_beentry` has been renamed to
`pgstat_get_local_beentry_by_index` in PG16.0

Relevant PG commit:
8dfa37b797
8dfa37b797843a83a5756ea3309055e8953e1a86

Sister PR
https://github.com/citusdata/the-process/pull/150
 2023-09-15 12:23:04 +03:00
Gürkan İndibay 7c0b289761
Adds alter database set option (#7181) 
DESCRIPTION: Adds support for ALTER DATABASE <db_name> SET .. statement
propagation
SET statements in Postgres has a common structure which is already being
used in Alter Function
statement. 
In this PR, I added a util file; citus_setutils and made it usable for
both for
alter database<db_name>set .. and alter function ... set ... statements.
With this PR, below statements will be propagated
```sql
ALTER DATABASE name SET configuration_parameter { TO | = } { value | DEFAULT }
ALTER DATABASE name SET configuration_parameter FROM CURRENT
ALTER DATABASE name RESET configuration_parameter
ALTER DATABASE name RESET ALL
```
Additionally, there was a bug in processing float values in the common
code block.
I fixed this one as well

Previous
```C
case T_Float:
			{
				appendStringInfo(buf, " %s", strVal(value));
				break;
			}
```
Now
```C
case T_Float:
			{
				appendStringInfo(buf, " %s", nodeToString(value));
				break;
			}
```
 2023-09-14 16:29:16 +03:00
aykut-bozkurt 26dc407f4a
bump citus and columnar into 12.2devel (#7200) 2023-09-14 12:03:09 +03:00
Gürkan İndibay e5e64b7454
Adds alter database propagation - with and refresh collation (#7172) 
DESCRIPTION: Adds ALTER DATABASE WITH ... and REFRESH COLLATION VERSION
support

This PR adds supports for basic ALTER DATABASE statements propagation 
support. Below statements are supported:
ALTER DATABASE <database_name> with IS_TEMPLATE <true/false>;
ALTER DATABASE <database_name> with CONNECTION LIMIT <integer_value>;
ALTER DATABASE <database_name> REFRESH COLLATION VERSION;

---------

Co-authored-by: Jelte Fennema-Nio <jelte.fennema@microsoft.com>
 2023-09-12 14:09:15 +03:00