Commit Graph

1408 Commits (8544878c4b2e752d2b03df47f5ee5f58a3b6bc5b)

Author SHA1 Message Date
Marco Slot df6d56c1ed Merge pull request #1606 from citusdata/fix_copy_dropped_columns
Consider dropped columns that precede the partition column in COPY
2017-08-22 13:13:09 +02:00
Marco Slot cf375d6a66 Consider dropped columns that precede the partition column in COPY 2017-08-22 13:02:35 +02:00
Marco Slot 15af3c5445 Merge pull request #1603 from citusdata/fix_lock_graph_allocs
Avoid overflowing PROCStack in BuildWaitGraphForSourceNode
2017-08-22 09:20:48 +02:00
Marco Slot bd6bf29983 Don't add procs multiple times in BuildWaitGraphForSourceNode 2017-08-21 16:48:30 +02:00
Önder Kalacı 734aeebc47 Merge pull request #1592 from citusdata/improve_maintanince_deamon
Terminate bg worker on drop database
2017-08-18 16:38:25 +03:00
Onder Kalaci 6532b69873 Kill the maintenance daemon on DROP DATABASE 2017-08-18 16:03:08 +03:00
Metin Döşlü b5109028bc Merge pull request #1598 from citusdata/fix_no_shards_bug
Fix a crash on zero-shard tables
2017-08-18 15:15:04 +03:00
Metin Doslu 0d052e9864 Fix a crash on zero-shard tables 2017-08-18 13:53:59 +03:00
Önder Kalacı 96391bea15 Merge pull request #1595 from citusdata/improve_deadlock_detection
Improve deadlock detection
2017-08-18 13:28:21 +03:00
Önder Kalacı b82f886ad3 Merge branch 'master' into improve_deadlock_detection 2017-08-18 13:07:18 +03:00
Marco Slot 2cc46f3a0c Merge pull request #1584 from citusdata/fix_drop_extension
Maintenance daemon ensures that the extension is valid
2017-08-18 11:32:44 +02:00
Marco Slot 7523753a73 Clear metadata OID cache prior to deadlock detection 2017-08-18 11:20:24 +02:00
Andres Freund b936bde936 Take AccessShareLock on the extension prior to running deadlock detection 2017-08-18 11:20:24 +02:00
Onder Kalaci 20679c9e8b Relax assertion on deadlock detection considering
self deadlocks.
2017-08-18 11:16:38 +03:00
Onder Kalaci 550a5578d8 Skip deadlock detection on the workers
Do not run distributed deadlock detection
on the worker nodes to prevent errornous
decisions to kill the deadlocks.
2017-08-17 19:43:38 +03:00
Burak Yucesoy 0ddcd726c9 Merge pull request #1464 from citusdata/copy_copy_isolation_test 2017-08-17 17:47:42 +03:00
Burak Yucesoy ae32d786cf Add new isolation tests 2017-08-17 17:46:03 +03:00
Marco Slot 131baeda3d Merge pull request #1585 from citusdata/exit_maintenanced
Maintenance daemon dies peacefully when it gets lost finding itself
2017-08-17 09:11:27 +02:00
Marco Slot 1eca53ad40 Exit maintenanced on database crash 2017-08-16 18:29:44 +02:00
Marco Slot a5d54382ef Merge pull request #1577 from citusdata/follower_get_active_worker_nodes
Return readable nodes in master_get_active_worker_nodes
2017-08-16 14:19:59 +02:00
Marco Slot 9e7b1fb858 Return readable nodes in master_get_active_worker_nodes 2017-08-16 11:28:47 +02:00
Hadi Moshayedi e5fbcf37dd Add Savepoint Support (#1539)
This change adds support for SAVEPOINT, ROLLBACK TO SAVEPOINT, and RELEASE SAVEPOINT.

When transaction connections are not established yet, savepoints are kept in a stack and sent to the worker when the connection is later established. After establishing connections, savepoint commands are sent as they arrive.

This change fixes #1493 .
2017-08-15 13:02:28 -04:00
Önder Kalacı dcabbc4a8e Merge pull request #1559 from citusdata/fix_deamon_upgrade
Add version check to the maintenance daemon
2017-08-15 19:24:18 +03:00
Onder Kalaci 205501532a Add version check to the maintenance daemon
We should prevent running the deadlock detection if
there is a major version change. Otherwise, the daemon
may access to obsolete metadata catalog tables.
2017-08-15 18:47:13 +03:00
Marco Slot 0d71fcd8af Merge pull request #1567 from citusdata/fix_2pc_issues
Fix 2pc issues
2017-08-15 14:22:47 +02:00
Marco Slot 3ff46245b3 Make sure we don't use 2PC in copy from worker 2017-08-15 13:44:20 +02:00
Marco Slot 4614814de1 Enable 2PC for INSERT...SELECT via coordinator 2017-08-15 13:44:20 +02:00
Marco Slot fa70089766 Enable 2PC during distributed table creation 2017-08-15 13:44:20 +02:00
Marco Slot 9232823070 Abort on failure on master connection during copy from worker 2017-08-15 13:44:20 +02:00
Marco Slot df7723cde5 Should not commit on aborted non-critical connections 2017-08-15 13:44:20 +02:00
Burak Yücesoy e14e5f0d25 Merge pull request #1566 from citusdata/switch_postgres_branch
Switch to Postgres REL_STABLE_10 branch
2017-08-15 14:39:52 +03:00
Burak Yucesoy cdcc5fdf65 Switch to Postgres REL_STABLE_10 branch
PostgreSQL master branch is now stamped with 11devel and we should
only use master branch if we want to test against PostgreSQL 11. For
PostgreSQL 9.6 tests we should use REL9_6_STABLE and for PostgreSQL 10
we should use REL_10_STABLE. v0.6.4 tag in our tools repo addresses
this problem.

Apart from that we may want to add PostgreSQL 11 to our test matrix soon.
v0.6.4 handles that too. We just need add PostgreSQL 11 to our test matrix
and stop erroring out if we are compiling Citus against PostgreSQL 11.
2017-08-15 14:25:29 +03:00
Eren Başak f1b51d7bbe Merge pull request #1551 from citusdata/fix_pg_worker_list_bug
Fix pg_worker_list use-after-free bug
2017-08-14 19:28:50 +03:00
Eren Başak 77626c4238 Fix NULL nodeClusterString crush on pg_worker_list.conf migrations 2017-08-14 18:13:53 +03:00
Eren Başak b3d2f9ba71 Fix pg_worker_list use-after-free bug
This change fixes a use-after-free bug while renaming obsolete
`pg_worker_list.conf` file, which causes Citus to crash during upgrade
(or even extension creation) if `pg_worker_list.conf` exists.
2017-08-14 18:13:53 +03:00
Burak Yücesoy b7e55e0c81 Merge pull request #1544 from citusdata/acquire_locks_for_partitioned_table_ops
Acquire proper locks for partitioned table operations
2017-08-14 15:09:03 +03:00
Burak Yucesoy 45b273321f Add tests for locking operations on partitioned tables 2017-08-14 14:55:45 +03:00
Burak Yucesoy dfdfb44ebf Acquire shard resource locks on parent tables while operating on partitions 2017-08-14 14:44:30 +03:00
Burak Yucesoy a321e750c0 Acquire relation locks on partitions while operation on parent table 2017-08-14 14:44:30 +03:00
Burak Yucesoy 52b9e35d50 Add relationIdList field to the Job struct 2017-08-14 14:06:22 +03:00
Önder Kalacı 45957e5688 Merge pull request #1529 from citusdata/deadlock_detection_main
Distributed Deadlock detection
2017-08-12 14:02:27 +03:00
Onder Kalaci 4f668ad38b Make the test outputs consistent
by using VACUUM ANALYZE on the tables.
2017-08-12 13:29:25 +03:00
Onder Kalaci 0ba2f9e4e4 Add regression tests for distributed deadlock detection 2017-08-12 13:29:25 +03:00
Onder Kalaci 5b48de7430 Improve deadlock detection for MX
We added a new field to the transaction id that is set to true only
for the transactions initialized on the coordinator. This is only
useful for MX in order to distinguish the transaction that started
the distributed transaction on the coordinator where we could
 have the same transactions' worker queries on the same node.
2017-08-12 13:28:37 +03:00
Onder Kalaci 59133415b0 Add logging infrasture for distributed deadlock detection
We added a new GUC citus.log_distributed_deadlock_detection
which is off by default. When set to on, we log some debug messages
related to the distributed deadlock to the server logs.
2017-08-12 13:28:37 +03:00
Onder Kalaci e5d5bdff51 Enable distributed deadlock detection on the maintenance deamon
With this commit, the maintenance deamon starts to check for
distributed deadlocks.

We also introduced a GUC variable (distributed_deadlock_detection_factor)
whose value is multiplied with Postgres' deadlock_timeout. Setting
it to -1 disables the distributed deadlock detection.
2017-08-12 13:28:37 +03:00
Onder Kalaci 66936053a0 Improve error messages when a backend is cancelled by deadlock detection
We send SIGINT to a backend that is cancelled due to a deadlock. That
approach ends up being a very confusing error message.

With this commit we intercept the error messages and show a more
meaningful error message to the user.
2017-08-12 13:28:37 +03:00
Onder Kalaci be4fc45c03 Deprecate enable_deadlock_prevention flag
Now that we already have the necessary infrastructure for detecting
distributed deadlocks. Thus, we don't need enable_deadlock_prevention
which is purely intended for preventing some forms of distributed
deadlocks.
2017-08-12 13:28:37 +03:00
Onder Kalaci a333c9f16c Add infrastructure for distributed deadlock detection
This commit adds all the necessary pieces to do the distributed
deadlock detection.

Each distributed transaction is already assigned with distributed
transaction ids introduced with
3369f3486f. The dependency among the
distributed transactions are gathered with
80ea233ec1.

With this commit, we implement a DFS (depth first seach) on the
dependency graph and search for cycles. Finding a cycle reveals
a distributed deadlock.

Once we find the deadlock, we examine the path that the cycle exists
and cancel the youngest distributed transaction.

Note that, we're not yet enabling the deadlock detection by default
with this commit.
2017-08-12 13:28:37 +03:00
Marco Slot d19818de21 Merge pull request #1543 from citusdata/test-follower-cluster
Add make target for testing follower clusters
2017-08-12 12:18:56 +02:00