Commit Graph

5584 Commits (34b17b1020860fe151c8c5318b39e1d261bee00c)

Author SHA1 Message Date
Onder Kalaci 34b17b1020 Fix isolation tester 2022-02-16 20:40:47 +01:00
Onder Kalaci d5484d4e69 Fix isolation tester 2022-02-16 20:39:13 +01:00
Onder Kalaci 75e19087b4 Fix isolation tester 2022-02-16 20:27:19 +01:00
Onder Kalaci 91073156c8 Fix isolation tester 2022-02-16 20:25:38 +01:00
Onder Kalaci bfd5ca0216 Fix isolation tester 2022-02-16 18:34:02 +01:00
Onder Kalaci 8b04e37b73 Fix isolation tester 2022-02-16 18:28:40 +01:00
Onder Kalaci 0d9bf7ad22 Fix isolation tester 2022-02-16 17:15:12 +01:00
Onder Kalaci 45b1d04850 Fix isolation tester 2022-02-16 17:07:33 +01:00
Onder Kalaci 3c5a146ae1 Properly set worker_query and use 2022-02-16 16:31:27 +01:00
Onder Kalaci 1a1b94aa24 follow feedback 2022-02-15 16:14:58 +01:00
Onder Kalaci 109d15d604 follow feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci 1684154245 follow feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci 3b7c801632 follow feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci be1e64da87 follow feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci 6179e6a5ac follow feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci c8cb25aab3 Address feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci f8ba6ade40 Address feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci 6c075b5b42 Address feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci 4fe4fa1562 Address feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci d77bf1e1ae Address feedback 2022-02-15 15:40:21 +01:00
Onder Kalaci d5bf975f31 fixup 2022-02-15 15:40:21 +01:00
Onder Kalaci 2c7040486f Adjust the helper UDFs for isolation tests
And add new tests to show citus_lock_wait can detect non-tx commands as well
2022-02-15 15:40:21 +01:00
Onder Kalaci a46e8f29d1 Use global pids in citus_lock_waits 2022-02-15 15:40:21 +01:00
Onder Kalaci d4b956c7f2 Use the optional APIs introduced for dump global/local wait edges
Do not change any behavior, simply use them. In the follow-up commits,
we'll use the API with different parameters.
2022-02-15 15:40:21 +01:00
Onder Kalaci 32b5cf257c Dumping wait edges becomes optionally scan all backends
Before this commit, dumping wait edges can only be used for
distributed deadlock detection purposes. With this commit,
we open the possibility that we can use it for any backend.
2022-02-15 15:40:21 +01:00
Önder Kalacı bf5aa1e223
Merge pull request #5711 from citusdata/clean_up_gpid
Prevent any monitoring view/udf to show already exited backends
2022-02-15 15:40:05 +01:00
Onder Kalaci abd5b1c506 Prevent any monitoring view/udf to show already exited backends
The low-level StoreAllActiveTransactions() function filters out
backends that exited.

Before this commit, if you run a pgbench, after that you'd still
see the backends show up:
```SQL
 select count(*) from get_global_active_transactions();
┌───────┐
│ count │
├───────┤
│   538 │
└───────┘
```

After this patch, only active backends show-up:

```SQL
 select count(*) from get_global_active_transactions();
┌───────┐
│ count │
├───────┤
│    72 │
└───────┘
```
2022-02-14 17:34:32 +01:00
Ahmet Gedemenli 0411a98c99
Refactor EnsureSequentialMode functions (#5704) 2022-02-14 18:38:21 +03:00
Gledis Zeneli badfd561b2
Prevent Citus table functions from being called on shards (Fix #5610) (#5694)
DESCRIPTION: Prevent Citus table functions from being called on shards

The operations that guard against using shards are:
* Create Local Table
* Create distributed table (which affects reference table creation as well).

* I used a `ErrorIfRaltionIsKnownShard` instead of `ErrorIfIllegallyChangingKnownShard`.
`ErrorIfIllegallyChangingKnownShard` allows the operation if `citus.enable_manual_changes_to_shards`,
but I am not sure if it ever makes sense to create a distributed, reference, or citus local table out of a shard.

I tried to go over the code to identify other UDF-s where shards could be illegaly changed, but I could not find any other.
My knowledge of the codebase is not solid enough for me to say for sure.

Fixes #5610
2022-02-14 16:06:48 +03:00
Gledis Zeneli 8a3544b4d9
Merge pull request #5695 from citusdata/update-contributing-md
* Adds installation of `mitmproxy`. I was getting this error from running regression tests:
	```
	Can't exec "mitmdump": No such file or directory at /home/glediszeneli/citus/src/test/regress/pg_regress_multi.pl line 215.
	```
* Add a comment to alternatively use `install-all` in the setup. Without `install-all` the `mutli-extension` regression test fails.
2022-02-14 15:17:41 +03:00
Gledis Zeneli b9dfeba050
Merge branch 'master' into update-contributing-md 2022-02-14 14:30:35 +03:00
Hanefi Onaldi 986d8cff49
Merge pull request #5682 from citusdata/metadata-iso-tests 2022-02-11 16:05:57 +03:00
gledis69 98f7c6bc49 Merge branch 'master' into update-contributing-md 2022-02-11 14:55:11 +03:00
gledis69 5478bd0105 Remove extra sudo in comment for Mac 2022-02-11 14:23:43 +03:00
gledis69 49c594a550 Adding install-all comment to all OS-es 2022-02-11 14:17:22 +03:00
Hanefi Onaldi 2e5ca8ba2b
Add isolation tests for metadata sync vs all
This commit introduces several test cases for concurrent operations that
change metadata, and a concurrent metadata sync operation.

The overall structure is as follows:
- Session#1 starts metadata syncing in a transaction block
- Session#2 does an operation that change metadata
- Both sessions are committed
- Another session checks whether the metadata are the same accross all
  nodes in the cluster.
2022-02-11 01:55:04 +03:00
Önder Kalacı dc6c194916
Show IDLE backends in citus_dist_stat_activity (#5700)
* Break the dependency to CitusInitiatedBackend infrastructure

With this change, we start to show non-distributed backends as well
in citus_dist_stat_activity. I think that
  (a) it is essential for making citus_lock_waits to work for blocked
      on DDL commands.
  (b) it is more expected from the user's perspective. The name of
      the view is a little inconsistent now (e.g., citus_dist_stat_activity)
      but we are already planning to improve the names with followup
      PRs.

Also, we have global pids assigned, the CitusInitiatedBackend
becomes obsolete.
2022-02-10 08:59:28 -08:00
Ahmet Gedemenli defc2d991f
Merge pull request #5663 from citusdata/schema-propagation
Propagate schema operations
2022-02-10 17:30:35 +03:00
Ahmet Gedemenli 76b63a307b Propagate create/drop schema commands 2022-02-10 14:58:09 +03:00
Marco Slot 04408cfded
Merge pull request #5693 from citusdata/marcocitus/from-pushdown 2022-02-09 21:14:45 +01:00
Marco Slot d0711ea9b4 Delegate function calls in FROM outside of transaction block 2022-02-09 20:56:25 +01:00
gledis69 4c2a0f0aa0 Removing install-all, but adding a comment about it 2022-02-09 22:00:39 +03:00
Önder Kalacı 60ce24578f
Merge pull request #5697 from citusdata/improve_nodecon_info
Prevent citus.node_conninfo to use "application_name"
2022-02-09 13:38:36 +01:00
Onder Kalaci 1c30f61a70 Prevent citus.node_conninfo to use "application_name"
With https://github.com/citusdata/citus/pull/5657, Citus uses
a fixed application_name while connecting to remote nodes
for internal purposes.

It means that we cannot allow users to override it via
citus.node_conninfo.
2022-02-09 13:22:04 +01:00
Teja Mupparti 1e3c8e34c0 Allow create_distributed_function() on a function owned by an extension
Implement #5649
Allow create_distributed_function() on functions owned by extensions

1) Only update pg_dist_object, and do not propagate CREATE FUNCTION.
2) Ensure corresponding extension is in pg_dist_object.
3) Verify if dependencies exist on the function they should resolve to the extension.
4) Impact on node-scaling: We build a list of ddl commands based on all objects in
   pg_dist_object. We need to omit the ddl's for the extension-function, as it
   will get propagated by the virtue of the extension creation.
5) Extra checks for functions coming from extensions, to not propagate changes
   via ddl commands, even though the function is marked as distributed in pg_dist_object
2022-02-08 11:52:56 -08:00
Halil Ozan Akgül 474e36a405
Merge pull request #5601 from citusdata/global_pid
Introduce global PID
2022-02-08 18:43:38 +03:00
Halil Ozan Akgul 8ee02b29d0 Introduce global PID 2022-02-08 16:49:38 +03:00
Burak Velioglu 6376eaf0e0
Merge pull request #5684 from citusdata/velioglu/superuser_connection_for_dep
Use super user connection while propagating dependent objects' pg_dist_object entries
2022-02-07 18:38:29 +03:00
gledis69 ed107835cb Updates a few details in Contributing.md
* Adds installation of `mitmproxy`. I was getting this error from running regression tests:
```
Can't exec "mitmdump": No such file or directory at /home/glediszeneli/citus/src/test/regress/pg_regress_multi.pl line 215.
```
* Calls `install-all` in the setup. Without `install-all` the `mutli-extension` regression test failed.
2022-02-07 18:34:39 +03:00
Burak Velioglu 0a70b78bf5
Add test for dist type 2022-02-07 17:50:49 +03:00