Commit Graph

5612 Commits (dda47dae7d8e2f2692f05925feb6104bdcca9b0a)

Author SHA1 Message Date
Önder Kalacı dda47dae7d
Merge pull request #5734 from citusdata/remove_citus_backend
Drop support for CitusInitiatedBackend
2022-02-24 15:29:19 +01:00
Onder Kalaci df95d59e33 Drop support for CitusInitiatedBackend
CitusInitiatedBackend was a pre-mature implemenation of the whole
GlobalPID infrastructure. We used it to track whether any individual
query is triggered by Citus or not.

As of now, after GlobalPID is already in place, we don't need
CitusInitiatedBackend, in fact it could even be wrong.
2022-02-24 12:12:43 +01:00
Marco Slot 9b4db12651
Merge pull request #5743 from citusdata/marcocitus/drop-wpqr 2022-02-24 10:55:37 +01:00
Marco Slot 0c4e3cb69c Drop worker_partition_query_result on downgrade 2022-02-24 10:18:56 +01:00
Hanefi Onaldi 1399853608
Merge pull request #5730 from citusdata/locks-on-ddltasklist 2022-02-24 03:47:30 +03:00
Hanefi Onaldi 7bd6c2c9ac
Isolation tests for various ddl operations and metadata sync 2022-02-24 03:19:56 +03:00
Hanefi Onaldi f4e8af2c22
Do not acquire locks on node metadata explicitly 2022-02-24 03:19:56 +03:00
Hanefi Onaldi b70949ae8c
Lock nodes when building ddl task lists 2022-02-24 03:19:56 +03:00
Marco Slot 955eabfcd6
Merge pull request #5400 from citusdata/marcocitus/repartition-using-intermediate-results 2022-02-23 20:03:42 +01:00
Marco Slot ef1ceb3953 Only use a single placement for map tasks 2022-02-23 19:40:21 +01:00
Marco Slot 8de802eec5 Enable local_shared_pool_size 5 in arbitrary configs test 2022-02-23 19:40:21 +01:00
Marco Slot 490765a754 Enable re-partition joins after local execution 2022-02-23 19:40:21 +01:00
Marco Slot 3cd9aa655a Stop using citus.binary_worker_copy_format 2022-02-23 19:40:21 +01:00
Marco Slot 5ac0d31e8b Fix re-partition hash range generation 2022-02-23 19:40:21 +01:00
Marco Slot 72d8fde28b Use intermediate results for re-partition joins 2022-02-23 19:40:21 +01:00
Nils Dijk 1fb970224e
Fix: partitioned index dependencies (#5741)
#5685 introduced the resolution of dependencies for indices. This missed support for indices on partitioned tables. This change adds support for partitioned indices to the dependency resolution code.
2022-02-23 17:53:26 +03:00
Jelte Fennema e1afd30263
Speed up test runs on WSL2 a lot (#5736)
It turns out `whereis` is incredibly slow on WSL2 (at least on my
machine):

```
$ time whereis diff
diff: /usr/bin/diff /usr/share/man/man1/diff.1.gz

real    0m0.408s
user    0m0.010s
sys 0m0.101s
```

This command is run by our custom `diff` script, which is run for every
test file that is run. So this adds lots of unnecessary runtime time to
tests.

This changes our custom `diff` script to only call `whereis` in the
strange case that `/usr/bin/diff` does not exist.

The impact of this small change on the total runtime of the tests on WSL
is huge. As an example the following command takes 18 seconds without
this change and 7 seconds with it:
```
make -C src/test/regress/ check-arbitrary-configs CONFIGS=PostgresConfig
```
2022-02-23 13:03:29 +01:00
Ahmet Gedemenli 9a8f11a086
Merge pull request #5572 from citusdata/add-citus-managed-tables-to-arbitrary-configs
Add use_citus_managed_tables to arbitrary configs
2022-02-22 11:54:45 +03:00
Ahmet Gedemenli 8b9402540f Add use_citus_managed_tables to arbitrary configs
(cherry picked from commit 4e93afd1f78854e1aaab63690c441b0b0598a82c)
(cherry picked from commit 0295fe2f5b)
(cherry picked from commit 878510725fab9cb6870b4504e0b1f055d7bbc68d)
2022-02-22 11:39:30 +03:00
Teja Mupparti a62901396b Allow unsafe triggers via a GUC 2022-02-21 22:45:17 -08:00
Önder Kalacı d0aa450d7b
Merge pull request #5716 from citusdata/improve_citus_lock_waits_and_use_worker_query
Properly set worker_query and use instead of application name on `citus_[dist/worker]_stat_activity`
2022-02-21 18:28:20 +01:00
Onder Kalaci 95d5918967 Properly set worker_query and use 2022-02-21 18:22:33 +01:00
Önder Kalacı 0dd2ddaa70
Merge pull request #5702 from citusdata/improve_citus_lock_waits
Improve citus lock waits to show non-tx blocked distributed processes as well
2022-02-21 18:00:51 +01:00
Onder Kalaci dffcafc096 Use global pids in citus_lock_waits 2022-02-21 17:46:34 +01:00
Onder Kalaci 331af3dce8 Dumping wait edges becomes optionally scan all backends
Before this commit, dumping wait edges can only be used for
distributed deadlock detection purposes. With this commit,
we open the possibility that we can use it for any backend.
2022-02-21 17:37:07 +01:00
Halil Ozan Akgül c866f4b2d6
Merge pull request #5699 from citusdata/global_cancellation
Overrides pg_cancel/terminate_backend functions to use gpid
2022-02-21 16:51:08 +03:00
Halil Ozan Akgul f6cd4d0f07 Overrides pg_cancel_backend and pg_terminate_backend to accept global pid 2022-02-21 16:41:35 +03:00
Ahmet Gedemenli 70dc85239f
Merge pull request #5727 from citusdata/check-distributed-first-for-DropSchemaStmts
Do distributed check first, for DropSchema stmts
2022-02-21 15:26:16 +03:00
Ahmet Gedemenli c1d5ca9896 Do distributed check first, for DropSchema stmts 2022-02-21 14:43:04 +03:00
Ahmet Gedemenli 24bdb287ed
Merge pull request #5723 from citusdata/refactor-distcol-for-createdistributedtable
Refactor CreateDistributedTable to take column name
2022-02-21 12:29:13 +03:00
Ahmet Gedemenli 28aa715ce2 Add test for citus local tables with dropped columns 2022-02-21 12:07:17 +03:00
Ahmet Gedemenli 2bc6a00408 Refactor CreateDistributedTable to take column name 2022-02-21 12:07:17 +03:00
Ying Xu 6d16e9ba56
Merge pull request #5688 from citusdata/maryxu/checkcitusversion_columnar
Copied CheckCitusVersion into Columnar and added new call site in beginscan_extended
2022-02-18 10:01:13 -08:00
yxu2162 8974b2de66 Copied CheckCitusVersion over to Columnar to handle dependency issue. If we split columnar into two extensions, this will later be changed tl CheckColumnarVersion. 2022-02-18 09:47:39 -08:00
Philip Dubé 854f6036a9
Merge pull request #5722 from citusdata/avoid-exceptional-control-flow-in-fluent-py
fluent.py: prefer simpler return based control flow in _accept rather than relying on raising an exception
2022-02-18 16:27:08 +00:00
Philip Dubé 3d044dc543
Merge branch 'master' into avoid-exceptional-control-flow-in-fluent-py 2022-02-18 16:10:45 +00:00
Burak Velioglu 259707b630
Merge pull request #5701 from citusdata/velioglu/function_propagation
Distribute functions with CREATE FUNCTION command
2022-02-18 17:57:53 +03:00
Burak Velioglu fa6866ed36
Start to propagate functions to worker nodes with
CREATE FUNCTION command together with it's dependencies.

If the function depends on any nondistributable object,
function will be created only locally. Parameterless
version of create_distributed_function becomes obsolete
with this change, it will deprecated from the code with a subsequent PR.
2022-02-18 13:56:51 +03:00
Gledis Zeneli 0ca060a820
Merge pull request #5713 from citusdata/prevent-deadlock-on-collation-create
* When a worker tried to create a collation which had a dependency in the same worker node,
it would cause a deadlock, now it throws the correct "not a coordinator" error.
2022-02-18 13:29:46 +03:00
gledis69 a14fada153 Prevent Deadlocks When a Worker Tries to Create Collation (Fix #5583)
* When a worker tried to create a collation which had a dependency in the same worker node,
it would cause a deadlock, now it throws the correct "not a coordinator" error.
2022-02-18 12:28:02 +03:00
Teja Mupparti 46fa47beea Force-delegated functions' distribution argument must be reset as soon as the routine completes execution,
and not wait until the top level Executor ends. This fixes issue #5687
2022-02-17 10:48:30 -08:00
Philip Dubé e4420a6252 fluent.py: prefer simpler return based control flow in _accept rather than relying on raising an exception 2022-02-17 13:30:17 +00:00
Nils Dijk 754d894375
Merge pull request #5721 from citusdata/fix/reuse-get-rolespec
reuse GetRoleSpecObjectForUser
2022-02-17 13:59:04 +01:00
Nils Dijk 768b320470
reuse GetRoleSpecObjectForUser 2022-02-17 13:16:10 +01:00
Nils Dijk ea86f9f94e
Add support for TEXT SEARCH CONFIGURATION objects (#5685)
DESCRIPTION: Implement TEXT SEARCH CONFIGURATION propagation

The change adds support to Citus for propagating TEXT SEARCH CONFIGURATION objects. TSConfig objects cannot always be created in one create statement, and instead require a create statement followed by many alter statements to get turned into the object they should represent.

To support this we add functionality to the worker to create or replace objects based on a list of statements. When the lists of the local object and the remote object correspond 1:1 we skip the creation of the object and simply mark it distributed. This is especially important for TSConfig objects as initdb pre-populates databases with a dozen configurations (for many different languages).

When the user creates a new TSConfig based on the copy of an existing configuration there is no direct link to the object copied from. Since there is no link we can't simply rely on propagating the dependencies to the worker and send a qualified
2022-02-17 13:12:46 +01:00
Hanefi Onaldi 886db667ee
Merge pull request #5717 from citusdata/fix-enterprise-merge-docs 2022-02-17 13:56:44 +03:00
Hanefi Onaldi 3ca2be85a7
Introduce new error message on enterprise merge checks 2022-02-17 13:48:36 +03:00
Hanefi Onaldi 78795251e1
Revert "Improve CI checks for enterprise merges on master (#4981)"
This reverts commit b649dffabd.
2022-02-17 13:48:36 +03:00
Hanefi Onaldi 8cfb93a662
Merge pull request #5718 from citusdata/fix-enterprise-merges 2022-02-17 13:44:46 +03:00
Hanefi Onaldi ccc4cc6bf0
Move test in isolation schedule to prevent failure
We check for metadata consistency across the cluster in the test
isolation_metadata_sync_vs_all. However, some earlier tests in
enterprise repo leave invalid pg_dist_node entries in the worker nodes
that have Oid values for already dropped role objects.

To remedy that, I suggest that we move the test to earlier in the
schedule, thereby making the tests pass for the time being. We should
later introduce metadata checking either in a new isolation test or by
moving this test later in the schedule. However, we should do that after
we fix the underlying issue.
2022-02-17 13:15:21 +03:00