Commit Graph

6877 Commits (86817ff094aee3aef01fd7cafa7547e6d723b899)

Author SHA1 Message Date
gurkanindibay 86817ff094 Fixes main merge errors 2024-02-15 13:23:03 +03:00
Gürkan İndibay 943aaf39b4
Merge branch 'main' into grant_role_2pc 2024-02-15 11:38:18 +03:00
Gürkan İndibay 59da0633bb
Fixes invalid grantor field parsing in grant role propagation (#7451)
DESCRIPTION: Resolves an issue that disrupts distributed GRANT
statements with the grantor option

In this issue 3 issues are being solved:
1.Correcting the erroneous appending of multiple granted by in the
deparser.
2Adding support for grantor (granted by) in grant role propagation.
3. Implementing grantor (granted by) support during the metadata sync
grant role propagation phase.

Limitations: Currently, the grantor must be created prior to the
metadata sync phase. During metadata sync, both the creation of the
grantor and the grants given by that role cannot be performed, as the
grantor role is not detected during the dependency resolution phase.

---------

Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-02-15 08:27:29 +00:00
Gürkan İndibay c665cb8af3
Adds changelog for 11.0.9,11.1.7,11.2.2,11.3.1,12.0.1,12.1.2 (#7507) 2024-02-14 08:40:28 +03:00
Ivan Vyazmitinov 2fae91c5df
Force LC_COLLATE=C for sort in check_gucs_are_alphabetically_sorted.sh (#7489)
Fixed gucs check, as described
[here](https://github.com/citusdata/citus/pull/7286#discussion_r1481049261)
2024-02-08 12:21:21 +01:00
Onur Tirtir 689c6897a4
Refactor CREATE / DROP database functions for better readability (#7486) 2024-02-08 01:55:50 +03:00
eaydingol f01c5f2593
Move remaining citus_internal functions (#7478)
Moves the following functions to the Citus internal schema: 

citus_internal_local_blocked_processes
citus_internal_global_blocked_processes
citus_internal_mark_node_not_synced
citus_internal_unregister_tenant_schema_globally
citus_internal_update_none_dist_table_metadata
citus_internal_update_placement_metadata
citus_internal_update_relation_colocation
citus_internal_start_replication_origin_tracking
citus_internal_stop_replication_origin_tracking
citus_internal_is_replication_origin_tracking_active


#7405

---------

Co-authored-by: Jelte Fennema-Nio <jelte.fennema@microsoft.com>
2024-02-07 16:58:17 +03:00
Filip Sedlák 6869b3ad10
Fail early when shard can't be safely moved to a new node (#7467)
DESCRIPTION: citus_move_shard_placement now fails early when shard
cannot be safely moved

The implementation is quite simplistic -
`citus_move_shard_placement(...)` will fail with an error if there's any
new node in the cluster that doesn't have reference tables yet.

It could have been finer-grained, i.e. erroring only when trying to move
a shard to an unitialized node. Looking at the related functions -
`replicate_reference_tables()` or `citus_rebalance_start()`, I think
it's acceptable behaviour. These other functions also treat "any"
unitialized node as a temporary anomaly.

Fixes #7426

---------

Co-authored-by: Jelte Fennema-Nio <jelte.fennema@microsoft.com>
2024-02-07 12:04:52 +00:00
Gürkan İndibay 4e5379cc8a
Merge branch 'main' into grant_role_2pc 2024-02-07 14:50:02 +03:00
Karina 9ff8436f14
Create directories and files with pg_file_create_mode and pg_dir_create_mode permissions (#7479)
Since Postgres commit da9b580d files and directories are supposed to
be created with pg_file_create_mode and pg_dir_create_mode permissions
when default permissions are expected.

This fixes a failure of one of the postgres tests:
If we create file add.conf containing
```
shared_preload_libraries='citus'
```
and run postgres tests
```
TEMP_CONFIG=/path/to/add.conf make installcheck -C src/bin/pg_ctl/
```
then 001_start_stop.pl fails with
```
.../data/base/pgsql_job_cache mode must be 0750
```
in the log.

In passing this also stops creating directories that we haven't used
since Citus 7.4

This change explicitely doesn't change permissions of certificates/keys
that we create.

---------

Co-authored-by: Karina Litskevich <litskevichkarina@gmail.com>
2024-02-07 12:48:31 +01:00
gurkanindibay 0466f60244 Fixes review notes 2024-02-07 14:12:09 +03:00
gurkanindibay 94450cd391 Fixes review notes 2024-02-07 14:07:17 +03:00
gurkanindibay 78d850950b Fixes indentation 2024-02-01 17:14:49 +03:00
gurkanindibay 6bff0ad924 Fixes review issues 2024-02-01 17:14:17 +03:00
gurkanindibay 91b1e8ab5e Merge branch 'grant_role_2pc' of https://github.com/citusdata/citus into grant_role_2pc 2024-02-01 11:44:11 +03:00
Gürkan İndibay 762c390801
Merge branch 'main' into grant_role_2pc 2024-02-01 11:43:58 +03:00
gurkanindibay 97c060b3ac Fixes review notes 2024-02-01 11:28:00 +03:00
eaydingol 594cb6f274
Move more citus internal functions (#7473)
Moves the following functions:

 citus_internal_delete_colocation_metadata 
 citus_internal_delete_partition_metadata 
 citus_internal_delete_placement_metadata 
 citus_internal_delete_shard_metadata 
 citus_internal_delete_tenant_schema
2024-01-31 23:00:04 +03:00
eaydingol d05174093b
Move citus internal functions (#7470)
Move more functions to citus_internal schema, the list:

citus_internal_add_placement_metadata
citus_internal_add_shard_metadata
citus_internal_add_tenant_schema
citus_internal_adjust_local_clock_to_remote
citus_internal_database_command

#7405
2024-01-31 11:45:19 +00:00
Onur Tirtir 3ce731d497
Make multi_metadata_sync runnable via run_test.py (#7472) 2024-01-31 09:50:16 +00:00
Onur Tirtir 6f43d5c02f
Enhance technical README for DDL propagation (#7471) 2024-01-31 10:30:14 +01:00
Onur Tirtir 5aedec4242
Improve error message for recursive CTEs (#7407)
Fixes #2870
2024-01-30 15:12:48 +00:00
eaydingol f6ea619e27
Move citus internal functions (#7466)
Move the following functions from pg_catalog to citus_internal:

citus_internal_add_object_metadata
citus_internal_add_partition_metadata


#7405
2024-01-30 12:27:10 +03:00
gurkanindibay 848cf685c2 Changes postprocess logic 2024-01-29 23:26:05 +03:00
Gürkan İndibay fb1030f238
Merge branch 'main' into grant_role_2pc 2024-01-29 22:11:55 +03:00
gurkanindibay 4553e1ea5b Fixes indentation 2024-01-29 17:57:09 +03:00
gurkanindibay 71a733cdec Fixes indentation 2024-01-29 17:50:52 +03:00
gurkanindibay 754d5899f2 Refactors MarkObjectDistributed logic 2024-01-29 17:50:27 +03:00
gurkanindibay e2e868166c Fixes naming suggestions 2024-01-29 16:34:37 +03:00
Gürkan İndibay 510e5581ad
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 16:33:49 +03:00
Gürkan İndibay 2f45493366
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 16:33:42 +03:00
Gürkan İndibay 42864d7069
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 16:33:27 +03:00
Gürkan İndibay 9632798dc5
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 16:32:54 +03:00
Gürkan İndibay 3533461712
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 13:16:27 +03:00
Gürkan İndibay 58bddca01c
Update src/backend/distributed/commands/utility_hook.c
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-29 13:16:14 +03:00
Onur Tirtir 9c243d4477
Improve check_gucs_are_alphabetically_sorted.sh (#7460)
Apparently https://github.com/citusdata/citus/pull/7452 was not enough,
need to consider the GUC-like expressions only within
RegisterCitusConfigVariables
function.
2024-01-26 12:10:35 +00:00
Gürkan İndibay 3189159703
Merge branch 'main' into grant_role_2pc 2024-01-26 15:02:52 +03:00
eaydingol 5d673874f7
Move citus internal functions (#7456)
Move citus_internal_acquire_citus_advisory_object_class_lock and
citus_internal_add_colocation_metadata functions from pg_catalog to
citus_internal.

#7405
2024-01-26 11:46:05 +03:00
Onur Tirtir 24188959ed
Improve the script that sorts GUCs in alphabetical order (#7452)
Soon we will have occurrences of "citus.X" in shared_library_init.c that
are not part of GUC defs, so we need to use a more precise regular
expression.
2024-01-25 11:22:39 +03:00
gurkanindibay eba3553ada Changes test order 2024-01-24 21:48:11 +03:00
gurkanindibay 016081a496 Fixes test error 2024-01-24 21:18:11 +03:00
Gürkan İndibay ee59e7eaf5
Merge branch 'main' into grant_role_2pc 2024-01-25 16:00:21 +03:00
gurkanindibay 9a0259cf3e Adds metadata sync tests 2024-01-24 17:47:39 +03:00
eaydingol 542212c3d8
Make citus_internal schema public (#7450)
DESCRIPTION: Makes citus_internal schema public



#7405
2024-01-24 17:11:10 +03:00
Onur Tirtir 3de5601bcc
Replace LOCAL_HOST_NAME with LocalHostName (#7449)
The only usages of LOCAL_HOST_NAME were in functions that are only used
during regression tests and in places where it was used incorrectly.
2024-01-24 13:50:39 +00:00
Onur Tirtir 1d096df7f4
Not use hardcoded LOCAL_HOST_NAME but citus.local_hostname to distinguish loopback connections (#7436)
Fixes a bug that breaks queries from non-maindbs when
citus.local_hostname is set to a value different than "localhost".

This is a very old bug doesn't cause a problem as long as Citus catalog
is available to FindWorkerNode(). And the catalog is always available
unless we're in non-main database, which might be the case on main but
not on older releases, hence not adding a `DESCRIPTION`. For this
reason, I don't see a reason to backport this.

Maybe we should totally refrain using LOCAL_HOST_NAME in all code-paths,
but not doing that in this PR as the other paths don't seem to be
breaking something that is user-facing.

```c
char *
GetAuthinfo(char *hostname, int32 port, char *user)
{
	char *authinfo = NULL;
	bool isLoopback = (strncmp(LOCAL_HOST_NAME, hostname, MAX_NODE_LENGTH) == 0 &&
					   PostPortNumber == port);

	if (IsTransactionState())
	{
		int64 nodeId = WILDCARD_NODE_ID;

		/* -1 is a special value for loopback connections (task tracker) */
		if (isLoopback)
		{
			nodeId = LOCALHOST_NODE_ID;
		}
		else
		{
			WorkerNode *worker = FindWorkerNode(hostname, port);
			if (worker != NULL)
			{
				nodeId = worker->nodeId;
			}
		}

		authinfo = GetAuthinfoViaCatalog(user, nodeId);
	}

	return (authinfo != NULL) ? authinfo : "";
}
```
2024-01-24 12:58:55 +00:00
Filip Sedlák 8b48d6ab02
Log username in the failed connection message (#7432)
This patch includes the username in the reported error message.
This makes debugging easier when certain commands open connections
as other users than the user that is executing the command.

```
monitora_snapshot=# SELECT citus_move_shard_placement(102030, 'monitora.db-dev-worker-a', 6005, 'monitora.db-dev-worker-a', 6017);
ERROR:  connection to the remote node monitora_user@monitora.db-dev-worker-a:6017 failed with the following error: fe_sendauth: no password supplied
Time: 40,198 ms
```
2024-01-24 11:24:23 +00:00
Halil Ozan Akgül 1cb2e1e4e8
Fixes create user queries from Citus non-main databases with other users (#7442)
This PR makes the connections to other nodes for
`mark_object_distributed` use the same user as
`execute_command_on_remote_nodes_as_user` so they'll use the same
connection.
2024-01-24 12:57:54 +03:00
Gokhan Gulbiz 3ffb831beb
Update contributing docs (#7447)
This is a minor change to use a generic name instead of our legacy CI
provider name in the contributing documentation.
2024-01-24 09:50:49 +01:00
Gürkan İndibay 863713e9b7
Refactors ExtendedTaskList methods (#7372)
ExecuteTaskListIntoTupleDestWithParam and ExecuteTaskListIntoTupleDest
are nearly the same. I parameterized and a made a reusable structure
here

---------

Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2024-01-24 06:00:19 +00:00