Commit Graph

5588 Commits (bd2072397a4484fe91aacd102d2bababf11f2970)

Author SHA1 Message Date
Burak Velioglu bd2072397a
Merge branch 'master' into velioglu/function_propagation 2022-02-16 00:09:06 +03:00
Burak Velioglu 24fe414eaa
Continue addressing reviews 2022-02-15 23:56:14 +03:00
Önder Kalacı bf5aa1e223
Merge pull request #5711 from citusdata/clean_up_gpid
Prevent any monitoring view/udf to show already exited backends
2022-02-15 15:40:05 +01:00
Burak Velioglu c7b8c4db34
Add all object dependency check 2022-02-15 17:37:48 +03:00
Burak Velioglu 110aff95c8
Address reviews 2022-02-15 00:59:19 +03:00
Onder Kalaci abd5b1c506 Prevent any monitoring view/udf to show already exited backends
The low-level StoreAllActiveTransactions() function filters out
backends that exited.

Before this commit, if you run a pgbench, after that you'd still
see the backends show up:
```SQL
 select count(*) from get_global_active_transactions();
┌───────┐
│ count │
├───────┤
│   538 │
└───────┘
```

After this patch, only active backends show-up:

```SQL
 select count(*) from get_global_active_transactions();
┌───────┐
│ count │
├───────┤
│    72 │
└───────┘
```
2022-02-14 17:34:32 +01:00
Ahmet Gedemenli 0411a98c99
Refactor EnsureSequentialMode functions (#5704) 2022-02-14 18:38:21 +03:00
Gledis Zeneli badfd561b2
Prevent Citus table functions from being called on shards (Fix #5610) (#5694)
DESCRIPTION: Prevent Citus table functions from being called on shards

The operations that guard against using shards are:
* Create Local Table
* Create distributed table (which affects reference table creation as well).

* I used a `ErrorIfRaltionIsKnownShard` instead of `ErrorIfIllegallyChangingKnownShard`.
`ErrorIfIllegallyChangingKnownShard` allows the operation if `citus.enable_manual_changes_to_shards`,
but I am not sure if it ever makes sense to create a distributed, reference, or citus local table out of a shard.

I tried to go over the code to identify other UDF-s where shards could be illegaly changed, but I could not find any other.
My knowledge of the codebase is not solid enough for me to say for sure.

Fixes #5610
2022-02-14 16:06:48 +03:00
Gledis Zeneli 8a3544b4d9
Merge pull request #5695 from citusdata/update-contributing-md
* Adds installation of `mitmproxy`. I was getting this error from running regression tests:
	```
	Can't exec "mitmdump": No such file or directory at /home/glediszeneli/citus/src/test/regress/pg_regress_multi.pl line 215.
	```
* Add a comment to alternatively use `install-all` in the setup. Without `install-all` the `mutli-extension` regression test fails.
2022-02-14 15:17:41 +03:00
Burak Velioglu afe8ce50c3
Test disabling for upgrade 2022-02-14 15:13:36 +03:00
Burak Velioglu ca12d09b8c
Citus indent 2022-02-14 14:32:23 +03:00
Gledis Zeneli b9dfeba050
Merge branch 'master' into update-contributing-md 2022-02-14 14:30:35 +03:00
Burak Velioglu c24a23b2f7
Fix citus owned test function 2022-02-14 14:28:47 +03:00
Burak Velioglu 38213e09ad
Add extension ownership check 2022-02-14 13:11:52 +03:00
Burak Velioglu b25a1c48e6
Revert it 2022-02-14 12:55:14 +03:00
Burak Velioglu 0557d67d78
Disable metadata sync for helper functions 2022-02-14 12:50:30 +03:00
Burak Velioglu 9574ebc988
Refactor 2022-02-14 12:18:24 +03:00
Burak Velioglu ad089f3dbb
Continue fixing tests 2022-02-14 11:32:29 +03:00
Burak Velioglu d7beb5a707
Add more tests 2022-02-14 00:57:20 +03:00
Burak Velioglu 9adef99248
Fix multi test 2022-02-14 00:32:37 +03:00
Burak Velioglu a829b7555b
Add function propagation tests 2022-02-14 00:18:42 +03:00
Burak Velioglu a58c15268b
Update function dep check 2022-02-13 17:17:46 +03:00
Burak Velioglu e249aa2d62
Merge branch 'master' into velioglu/function_propagation 2022-02-13 00:12:34 +03:00
Burak Velioglu 23aa73c988
Fix multi-1 tests 2022-02-12 23:40:20 +03:00
Hanefi Onaldi 986d8cff49
Merge pull request #5682 from citusdata/metadata-iso-tests 2022-02-11 16:05:57 +03:00
gledis69 98f7c6bc49 Merge branch 'master' into update-contributing-md 2022-02-11 14:55:11 +03:00
gledis69 5478bd0105 Remove extra sudo in comment for Mac 2022-02-11 14:23:43 +03:00
gledis69 49c594a550 Adding install-all comment to all OS-es 2022-02-11 14:17:22 +03:00
Hanefi Onaldi 2e5ca8ba2b
Add isolation tests for metadata sync vs all
This commit introduces several test cases for concurrent operations that
change metadata, and a concurrent metadata sync operation.

The overall structure is as follows:
- Session#1 starts metadata syncing in a transaction block
- Session#2 does an operation that change metadata
- Both sessions are committed
- Another session checks whether the metadata are the same accross all
  nodes in the cluster.
2022-02-11 01:55:04 +03:00
Önder Kalacı dc6c194916
Show IDLE backends in citus_dist_stat_activity (#5700)
* Break the dependency to CitusInitiatedBackend infrastructure

With this change, we start to show non-distributed backends as well
in citus_dist_stat_activity. I think that
  (a) it is essential for making citus_lock_waits to work for blocked
      on DDL commands.
  (b) it is more expected from the user's perspective. The name of
      the view is a little inconsistent now (e.g., citus_dist_stat_activity)
      but we are already planning to improve the names with followup
      PRs.

Also, we have global pids assigned, the CitusInitiatedBackend
becomes obsolete.
2022-02-10 08:59:28 -08:00
Ahmet Gedemenli defc2d991f
Merge pull request #5663 from citusdata/schema-propagation
Propagate schema operations
2022-02-10 17:30:35 +03:00
Burak Velioglu c78ca8cbad
Continue fixing multi tests 2022-02-10 16:59:08 +03:00
Ahmet Gedemenli 76b63a307b Propagate create/drop schema commands 2022-02-10 14:58:09 +03:00
Burak Velioglu cb4e53ad6e
Fix mx tests 2022-02-10 13:09:30 +03:00
Burak Velioglu 814e1a95de
Fix mx tests 2022-02-10 01:54:35 +03:00
Burak Velioglu 0cfe2375b6
Fix operation tests 2022-02-10 01:38:44 +03:00
Burak Velioglu 6de2d7f050
Fix check_multi tests 2022-02-10 01:20:42 +03:00
Burak Velioglu 8901936e9a
Check whether the function call is in transaction block 2022-02-10 00:04:51 +03:00
Marco Slot 04408cfded
Merge pull request #5693 from citusdata/marcocitus/from-pushdown 2022-02-09 21:14:45 +01:00
Marco Slot d0711ea9b4 Delegate function calls in FROM outside of transaction block 2022-02-09 20:56:25 +01:00
gledis69 4c2a0f0aa0 Removing install-all, but adding a comment about it 2022-02-09 22:00:39 +03:00
Burak Velioglu 5c74514600
Update tables type check 2022-02-09 18:10:47 +03:00
Burak Velioglu 9e938ba1a3
Style 2022-02-09 16:46:52 +03:00
Burak Velioglu 6770b222d5
Mark functions distributed if propagated to workers 2022-02-09 16:40:52 +03:00
Önder Kalacı 60ce24578f
Merge pull request #5697 from citusdata/improve_nodecon_info
Prevent citus.node_conninfo to use "application_name"
2022-02-09 13:38:36 +01:00
Onder Kalaci 1c30f61a70 Prevent citus.node_conninfo to use "application_name"
With https://github.com/citusdata/citus/pull/5657, Citus uses
a fixed application_name while connecting to remote nodes
for internal purposes.

It means that we cannot allow users to override it via
citus.node_conninfo.
2022-02-09 13:22:04 +01:00
Burak Velioglu a29748db0e
Add function for dependency check 2022-02-09 14:55:22 +03:00
Burak Velioglu c4fddf1406
Distribute functions with CREATE FUNCTION command 2022-02-09 13:46:26 +03:00
Teja Mupparti 1e3c8e34c0 Allow create_distributed_function() on a function owned by an extension
Implement #5649
Allow create_distributed_function() on functions owned by extensions

1) Only update pg_dist_object, and do not propagate CREATE FUNCTION.
2) Ensure corresponding extension is in pg_dist_object.
3) Verify if dependencies exist on the function they should resolve to the extension.
4) Impact on node-scaling: We build a list of ddl commands based on all objects in
   pg_dist_object. We need to omit the ddl's for the extension-function, as it
   will get propagated by the virtue of the extension creation.
5) Extra checks for functions coming from extensions, to not propagate changes
   via ddl commands, even though the function is marked as distributed in pg_dist_object
2022-02-08 11:52:56 -08:00
Halil Ozan Akgül 474e36a405
Merge pull request #5601 from citusdata/global_pid
Introduce global PID
2022-02-08 18:43:38 +03:00