Commit Graph

5444 Commits (29dd7dfe052dac3283af309b4f6f6daed18a5a63)

Author SHA1 Message Date
Gürkan İndibay 29dd7dfe05
Adds stackoverflow badge into README.md (#5589) 2022-01-04 10:42:18 +03:00
Halil Ozan Akgül 9cccaa11d3
Merge pull request #5587 from citusdata/add_isolation_check_mx
Add isolation_check_mx test
2021-12-30 15:31:58 +03:00
Halil Ozan Akgul 9547228e8d Add isolation_check_mx test 2021-12-30 14:58:30 +03:00
Halil Ozan Akgül 41b4462f6e
Merge pull request #5580 from citusdata/fix_metadata_sync_fails_on_multi_transaction_recovery
Fix metadata sync fails on multi_transaction_recovery
2021-12-29 11:35:05 +03:00
Halil Ozan Akgul aef2d83c7d Fix metadata sync fails on multi_transaction_recovery 2021-12-29 11:21:32 +03:00
Önder Kalacı d33650d1c1
Record if any partitioned Citus tables during upgrade (#5555)
With Citus 11, the default behavior is to sync the metadata.
However, partitioned tables created pre-Citus 11 might have
index names that are not compatiable with metadata syncing.

See https://github.com/citusdata/citus/issues/4962 for the
details.

With this commit, we record the existence of partitioned tables
such that we can fix it later if any exists.
2021-12-27 03:33:34 -08:00
Halil Ozan Akgül c43b6613d0
Merge pull request #5576 from citusdata/fix_metadata_sync_fails_on_multi_truncate
Fix metadata sync fails on multi_truncate
2021-12-27 14:08:18 +03:00
Halil Ozan Akgul 0c292a74f5 Fix metadata sync fails on multi_truncate 2021-12-27 13:54:53 +03:00
Önder Kalacı c9127f921f
Avoid round trips while fixing index names (#5549)
With this commit, fix_partition_shard_index_names()
works significantly faster.

For example,

32 shards, 365 partitions, 5 indexes drop from ~120 seconds to ~44 seconds
32 shards, 1095 partitions, 5 indexes drop from ~600 seconds to ~265 seconds

`queryStringList` can be really long, because it may contain #partitions * #indexes entries.

Before this change, we were actually going through the executor where each command
in the query string triggers 1 round trip per entry in queryStringList.

The aim of this commit is to avoid the round-trips by creating a single query string.

I first simply tried sending `q1;q2;..;qn` . However, the executor is designed to
handle `q1;q2;..;qn` type of query executions via the infrastructure mentioned
above (e.g., by tracking the query indexes in the list and doing 1 statement
per round trip).

One another option could have been to change the executor such that only track
the query index when `queryStringList` is provided not with queryString
including multiple `;`s . That is (a) more work (b) could cause weird edge
cases with failure handling (c) felt like coding a special case in to the executor
2021-12-27 10:29:37 +01:00
Halil Ozan Akgül e3d1a42f81
Merge pull request #5574 from citusdata/fix_metadata_sync_fails_on_multi_function_evaluation
Fix metadata sync fails on multi_function_evaluation
2021-12-27 10:53:04 +03:00
Halil Ozan Akgul bb636e6a29 Fix metadata sync fails on multi_function_evaluation 2021-12-24 19:32:58 +03:00
Halil Ozan Akgül 7e851d2b9b
Merge pull request #5553 from citusdata/fix_metadata_sync_fails_on_multi_sequence_default
Fix metadata sync fails on multi_sequence_default and multi_name_lengths
2021-12-24 17:33:37 +03:00
Halil Ozan Akgul 70e68d5312 Fix metadata sync fails on multi_name_lengths 2021-12-24 14:33:32 +03:00
Halil Ozan Akgul 5c2fb06322 Fix metadata sync fails on multi_sequence_default 2021-12-24 14:33:32 +03:00
Hanefi Onaldi caffaa8517
Merge pull request #5561 from citusdata/improve-circleci-config 2021-12-24 13:20:50 +03:00
Hanefi Onaldi 9ad29e5a9d
Improve CircleCI configs
- Parameterize PG versions
- Use single parens for consistency
- Fix spacing issues
- Remove unsupported attributes
- Compact check-style steps
- Parameterize PG versions for upgrade tests
2021-12-24 13:10:27 +03:00
Halil Ozan Akgül 7f82400352
Merge pull request #5558 from citusdata/turn_metadata_sync_on_in_multi_metadata_sync
Turn metadata sync on in multi_metadata_sync
2021-12-24 11:05:25 +03:00
Halil Ozan Akgul b9c06a6762 Turn metadata sync on in multi_metadata_sync 2021-12-24 10:58:13 +03:00
Hanefi Onaldi 479b2da740 Fix one flaky failure test 2021-12-23 20:11:45 +03:00
Ahmet Gedemenli 63626d4995
Merge pull request #5468 from citusdata/propagate-foreign-server-ops
Propagate foreign server ops
2021-12-23 19:04:52 +03:00
Ahmet Gedemenli 042d45b263 Propagate foreign server ops 2021-12-23 17:54:04 +03:00
Onur Tirtir 61b5fb1cfc
Run failure_test_helpers in base schedule (#5559) 2021-12-23 12:54:12 +01:00
Talha Nisanci e196d23854
Refactor AttributeEquivalenceId (#5006) 2021-12-23 13:19:02 +03:00
Hanefi Onaldi 76176caea7 Fix typo s/exlusive/exclusive/ 2021-12-23 01:35:01 +03:00
Hanefi Onaldi 1af8ca8f7c
Fix statical analysis findings (#5550) 2021-12-22 18:16:11 +03:00
Ahmet Gedemenli d5a969b055
Merge pull request #5540 from citusdata/fix-function-signature-generation
Fix function signature generation
2021-12-21 19:25:41 +03:00
Ahmet Gedemenli 8e4ff34a2e Do not include return table params in the function arg list
(cherry picked from commit 90928cfd74)

Fix function signature generation

Fix comment typo

Add test for worker_create_or_replace_object

Add test for recreating distributed functions with OUT/TABLE params

Add test for recreating distributed function that returns setof int

Fix test output

Fix comment
2021-12-21 19:01:42 +03:00
Marco Slot 80f41e94c0
Merge pull request #4945 from citusdata/marcocitus/set-transaction 2021-12-18 11:38:37 +01:00
Marco Slot 2eef71ccab Propagate SET TRANSACTION commands 2021-12-18 11:31:39 +01:00
Halil Ozan Akgül 2108410a40
Merge pull request #5535 from citusdata/turn_metadata_sync_on_in_add_corrdinator
Turn metadata sync on in add_coordinator, foreign_key_to_reference_ta…
2021-12-17 18:05:43 +03:00
Halil Ozan Akgul 46f718c76d Turn metadata sync on in add_coordinator, foreign_key_to_reference_table and replicate_reference_tables_to_coordinator 2021-12-17 16:33:25 +03:00
Halil Ozan Akgül c3195f75a5
Merge pull request #5545 from citusdata/turn_ddl_propagation_off_on_multi_copy
Turn ddl propagation off in worker on multi_copy
2021-12-17 16:29:13 +03:00
Halil Ozan Akgul 25755a7094 Turn ddl propagation off in worker on multi_copy 2021-12-17 15:54:20 +03:00
Önder Kalacı 695653911a
Merge pull request #4634 from citusdata/citus_grep_command
Grep Remote/Local commands
2021-12-17 12:07:45 +01:00
Onder Kalaci fc98f83af2 Add citus.grep_remote_commands
Simply applies

```SQL
SELECT textlike(command, citus.grep_remote_commands)
```
And, if returns true, the command is logged. Else, the log is ignored.

When citus.grep_remote_commands is empty string, all commands are
logged.
2021-12-17 11:47:40 +01:00
Halil Ozan Akgül 5b1a25ae7c
Merge pull request #5544 from citusdata/turn_metadata_sync_on_in_multi_replicate_reference_table
Turn metadata sync on in multi_replicate_reference_table and multi_citus_tools
2021-12-17 11:04:41 +03:00
Halil Ozan Akgul df8d0f3db1 Turn metadata sync on in multi_replicate_reference_table and multi_citus_tools 2021-12-17 10:25:57 +03:00
Onur Tirtir cc4c83b1e5
HAVE_LZ4 -> HAVE_CITUS_LZ4 (#5541) 2021-12-16 16:21:52 +03:00
Talha Nisanci c0945d88de
Normalize a debug failure to WARNING failure (#4996) 2021-12-16 13:43:49 +03:00
Halil Ozan Akgül 7d0f4f11c3
Merge pull request #5537 from citusdata/turn_metadata_sync_on_in_mx_regular_user
Turn metadata sync on in mx_regular_user and remove_coordinator
2021-12-16 11:35:08 +03:00
Halil Ozan Akgul 8943d7b52f Turn metadata sync on in mx_regular_user and remove_coordinator 2021-12-16 11:26:24 +03:00
Halil Ozan Akgül 047ae2cad0
Merge pull request #5534 from citusdata/turn_metadata_sync_on_in_multi_unsupported_worker_operations
Turn metadata sync on in multi_size_queries, multi_drop_extension and multi_unsupported_worker_operations
2021-12-16 11:25:19 +03:00
Halil Ozan Akgul b82af4db3b Turn metadata sync on in multi_size_queries, multi_drop_extension and multi_unsupported_worker_operations 2021-12-16 11:10:54 +03:00
Hanefi Onaldi 9d4d73898a
Move healthcheck logic into new file (#5531)
and add a missing `CheckCitusVersion(ERROR)` call
2021-12-15 15:58:20 -08:00
Hanefi Onaldi acdcd9422c
Fix one flaky failure test (#5528)
Removes flaky test
2021-12-15 18:59:58 +03:00
Hanefi Onaldi 29e4516642 Introduce citus_check_cluster_node_health UDF
This UDF coordinates connectivity checks accross the whole cluster.

This UDF gets the list of active readable nodes in the cluster, and
coordinates all connectivity checks in sequential order.

The algorithm is:

for sourceNode in activeReadableWorkerList:
    c = connectToNode(sourceNode)
    for targetNode in activeReadableWorkerList:
        result = c.execute(
            "SELECT citus_check_connection_to_node(targetNode.name,
                                                   targetNode.port")
        emit sourceNode.name,
             sourceNode.port,
             targetNode.name,
             targetNode.port,
             result

- result -> true  ->  connection attempt from source to target succeeded
- result -> false -> connection attempt from source to target failed
- result -> NULL  -> connection attempt from the current node to source node failed

I suggest you use the following query to get an overview on the connectivity:

SELECT bool_and(COALESCE(result, false))
FROM citus_check_cluster_node_health();

Whenever this query returns false, there is a connectivity issue, check in detail.
2021-12-15 01:41:51 +03:00
Hanefi Onaldi 13fff9c37a Remove NOOP tuplestore_donestoring calls
PostgreSQL does not need calling this function since 7.4 release, and it
is a NOOP.

For more details, check PostgreSQL commit below :

commit dd04e958c8b03c0f0512497651678c7816af3198
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sun Mar 9 03:34:10 2003 +0000

    tuplestore_donestoring() isn't needed anymore, but provide a no-op
    macro definition so as not to create compatibility problems.

diff --git a/src/include/utils/tuplestore.h b/src/include/utils/tuplestore.h
index b46babacd1..76fe9fb428 100644
--- a/src/include/utils/tuplestore.h
+++ b/src/include/utils/tuplestore.h
@@ -17,7 +17,7 @@
  * Portions Copyright (c) 1996-2002, PostgreSQL Global Development Group
  * Portions Copyright (c) 1994, Regents of the University of California
  *
- * $Id: tuplestore.h,v 1.8 2003/03/09 02:19:13 tgl Exp $
+ * $Id: tuplestore.h,v 1.9 2003/03/09 03:34:10 tgl Exp $
  *
  *-------------------------------------------------------------------------
  */
@@ -41,6 +41,9 @@ extern Tuplestorestate *tuplestore_begin_heap(bool randomAccess,

 extern void tuplestore_puttuple(Tuplestorestate *state, void *tuple);

+/* tuplestore_donestoring() used to be required, but is no longer used */
+#define tuplestore_donestoring(state)  ((void) 0)
+
 /* backwards scan is only allowed if randomAccess was specified 'true' */
 extern void *tuplestore_gettuple(Tuplestorestate *state, bool forward,
                                        bool *should_free);
2021-12-14 18:55:02 +03:00
Halil Ozan Akgül 1c5430635d
Merge pull request #5525 from citusdata/only_drop_dist_indexes_on_metadata_synced_nodes
Fix drop index trying to drop coordinator local indexes on metadata worker nodes
2021-12-14 15:37:45 +03:00
Halil Ozan Akgul e060720370 Fix metadata sync fails in multi_index_statements 2021-12-14 11:28:08 +03:00
Halil Ozan Akgul a951e52ce8 Fix drop index trying to drop coordinator local indexes on metadata worker nodes 2021-12-14 11:28:08 +03:00