Commit Graph

1394 Commits (a1387f4aa814afcc8d26cfec36d2006a486d18fa)

Author SHA1 Message Date
Onder Kalaci 550a5578d8 Skip deadlock detection on the workers
Do not run distributed deadlock detection
on the worker nodes to prevent errornous
decisions to kill the deadlocks.
2017-08-17 19:43:38 +03:00
Burak Yucesoy 0ddcd726c9 Merge pull request #1464 from citusdata/copy_copy_isolation_test 2017-08-17 17:47:42 +03:00
Burak Yucesoy ae32d786cf Add new isolation tests 2017-08-17 17:46:03 +03:00
Marco Slot 131baeda3d Merge pull request #1585 from citusdata/exit_maintenanced
Maintenance daemon dies peacefully when it gets lost finding itself
2017-08-17 09:11:27 +02:00
Marco Slot 1eca53ad40 Exit maintenanced on database crash 2017-08-16 18:29:44 +02:00
Marco Slot a5d54382ef Merge pull request #1577 from citusdata/follower_get_active_worker_nodes
Return readable nodes in master_get_active_worker_nodes
2017-08-16 14:19:59 +02:00
Marco Slot 9e7b1fb858 Return readable nodes in master_get_active_worker_nodes 2017-08-16 11:28:47 +02:00
Hadi Moshayedi e5fbcf37dd Add Savepoint Support (#1539)
This change adds support for SAVEPOINT, ROLLBACK TO SAVEPOINT, and RELEASE SAVEPOINT.

When transaction connections are not established yet, savepoints are kept in a stack and sent to the worker when the connection is later established. After establishing connections, savepoint commands are sent as they arrive.

This change fixes #1493 .
2017-08-15 13:02:28 -04:00
Önder Kalacı dcabbc4a8e Merge pull request #1559 from citusdata/fix_deamon_upgrade
Add version check to the maintenance daemon
2017-08-15 19:24:18 +03:00
Onder Kalaci 205501532a Add version check to the maintenance daemon
We should prevent running the deadlock detection if
there is a major version change. Otherwise, the daemon
may access to obsolete metadata catalog tables.
2017-08-15 18:47:13 +03:00
Marco Slot 0d71fcd8af Merge pull request #1567 from citusdata/fix_2pc_issues
Fix 2pc issues
2017-08-15 14:22:47 +02:00
Marco Slot 3ff46245b3 Make sure we don't use 2PC in copy from worker 2017-08-15 13:44:20 +02:00
Marco Slot 4614814de1 Enable 2PC for INSERT...SELECT via coordinator 2017-08-15 13:44:20 +02:00
Marco Slot fa70089766 Enable 2PC during distributed table creation 2017-08-15 13:44:20 +02:00
Marco Slot 9232823070 Abort on failure on master connection during copy from worker 2017-08-15 13:44:20 +02:00
Marco Slot df7723cde5 Should not commit on aborted non-critical connections 2017-08-15 13:44:20 +02:00
Burak Yücesoy e14e5f0d25 Merge pull request #1566 from citusdata/switch_postgres_branch
Switch to Postgres REL_STABLE_10 branch
2017-08-15 14:39:52 +03:00
Burak Yucesoy cdcc5fdf65 Switch to Postgres REL_STABLE_10 branch
PostgreSQL master branch is now stamped with 11devel and we should
only use master branch if we want to test against PostgreSQL 11. For
PostgreSQL 9.6 tests we should use REL9_6_STABLE and for PostgreSQL 10
we should use REL_10_STABLE. v0.6.4 tag in our tools repo addresses
this problem.

Apart from that we may want to add PostgreSQL 11 to our test matrix soon.
v0.6.4 handles that too. We just need add PostgreSQL 11 to our test matrix
and stop erroring out if we are compiling Citus against PostgreSQL 11.
2017-08-15 14:25:29 +03:00
Eren Başak f1b51d7bbe Merge pull request #1551 from citusdata/fix_pg_worker_list_bug
Fix pg_worker_list use-after-free bug
2017-08-14 19:28:50 +03:00
Eren Başak 77626c4238 Fix NULL nodeClusterString crush on pg_worker_list.conf migrations 2017-08-14 18:13:53 +03:00
Eren Başak b3d2f9ba71 Fix pg_worker_list use-after-free bug
This change fixes a use-after-free bug while renaming obsolete
`pg_worker_list.conf` file, which causes Citus to crash during upgrade
(or even extension creation) if `pg_worker_list.conf` exists.
2017-08-14 18:13:53 +03:00
Burak Yücesoy b7e55e0c81 Merge pull request #1544 from citusdata/acquire_locks_for_partitioned_table_ops
Acquire proper locks for partitioned table operations
2017-08-14 15:09:03 +03:00
Burak Yucesoy 45b273321f Add tests for locking operations on partitioned tables 2017-08-14 14:55:45 +03:00
Burak Yucesoy dfdfb44ebf Acquire shard resource locks on parent tables while operating on partitions 2017-08-14 14:44:30 +03:00
Burak Yucesoy a321e750c0 Acquire relation locks on partitions while operation on parent table 2017-08-14 14:44:30 +03:00
Burak Yucesoy 52b9e35d50 Add relationIdList field to the Job struct 2017-08-14 14:06:22 +03:00
Önder Kalacı 45957e5688 Merge pull request #1529 from citusdata/deadlock_detection_main
Distributed Deadlock detection
2017-08-12 14:02:27 +03:00
Onder Kalaci 4f668ad38b Make the test outputs consistent
by using VACUUM ANALYZE on the tables.
2017-08-12 13:29:25 +03:00
Onder Kalaci 0ba2f9e4e4 Add regression tests for distributed deadlock detection 2017-08-12 13:29:25 +03:00
Onder Kalaci 5b48de7430 Improve deadlock detection for MX
We added a new field to the transaction id that is set to true only
for the transactions initialized on the coordinator. This is only
useful for MX in order to distinguish the transaction that started
the distributed transaction on the coordinator where we could
 have the same transactions' worker queries on the same node.
2017-08-12 13:28:37 +03:00
Onder Kalaci 59133415b0 Add logging infrasture for distributed deadlock detection
We added a new GUC citus.log_distributed_deadlock_detection
which is off by default. When set to on, we log some debug messages
related to the distributed deadlock to the server logs.
2017-08-12 13:28:37 +03:00
Onder Kalaci e5d5bdff51 Enable distributed deadlock detection on the maintenance deamon
With this commit, the maintenance deamon starts to check for
distributed deadlocks.

We also introduced a GUC variable (distributed_deadlock_detection_factor)
whose value is multiplied with Postgres' deadlock_timeout. Setting
it to -1 disables the distributed deadlock detection.
2017-08-12 13:28:37 +03:00
Onder Kalaci 66936053a0 Improve error messages when a backend is cancelled by deadlock detection
We send SIGINT to a backend that is cancelled due to a deadlock. That
approach ends up being a very confusing error message.

With this commit we intercept the error messages and show a more
meaningful error message to the user.
2017-08-12 13:28:37 +03:00
Onder Kalaci be4fc45c03 Deprecate enable_deadlock_prevention flag
Now that we already have the necessary infrastructure for detecting
distributed deadlocks. Thus, we don't need enable_deadlock_prevention
which is purely intended for preventing some forms of distributed
deadlocks.
2017-08-12 13:28:37 +03:00
Onder Kalaci a333c9f16c Add infrastructure for distributed deadlock detection
This commit adds all the necessary pieces to do the distributed
deadlock detection.

Each distributed transaction is already assigned with distributed
transaction ids introduced with
3369f3486f. The dependency among the
distributed transactions are gathered with
80ea233ec1.

With this commit, we implement a DFS (depth first seach) on the
dependency graph and search for cycles. Finding a cycle reveals
a distributed deadlock.

Once we find the deadlock, we examine the path that the cycle exists
and cancel the youngest distributed transaction.

Note that, we're not yet enabling the deadlock detection by default
with this commit.
2017-08-12 13:28:37 +03:00
Marco Slot d19818de21 Merge pull request #1543 from citusdata/test-follower-cluster
Add make target for testing follower clusters
2017-08-12 12:18:56 +02:00
Marco Slot 59e626d158 Add regression tests for follower clusters 2017-08-12 12:05:56 +02:00
Marco Slot 55992d4bc0 Disallow task-tracker queries on follower clusters 2017-08-12 11:47:31 +02:00
Marco Slot c097bc9a01 Merge pull request #1503 from citusdata/fix_drop_create_deadlock
Fix drop table - create_distributed_table deadlock
2017-08-11 12:46:51 +02:00
velioglu 100739f62a Change citus subversion 2017-08-11 11:57:57 +03:00
Marco Slot 53584affa8 Fix locking in create_distributed_table 2017-08-11 11:34:33 +03:00
velioglu 7c65001e23 Do not delete row from colocation table within drop table 2017-08-11 11:34:33 +03:00
Burak Velioglu c4eb6c5153 Merge pull request #1532 from citusdata/subquery_pushdown_on_reference_tables
Subquery pushdown on reference tables
2017-08-11 10:32:22 +03:00
velioglu b0efffae1c Correct planner and add more tests 2017-08-11 10:16:13 +03:00
velioglu 7550b8ad52 Fix anchor shard id selection when reference table exists 2017-08-11 10:09:47 +03:00
velioglu ceba81ce35 Move physical planner checks to logical planner 2017-08-11 10:09:47 +03:00
velioglu 0359d03530 Add set operation check for reference tables 2017-08-11 10:09:47 +03:00
velioglu c4e3b8b5e1 Add planner changes and tests for subquery on reference tables 2017-08-11 10:09:47 +03:00
velioglu 45717dd013 Check equivalence on reference tables for subquery pushdown 2017-08-11 10:09:47 +03:00
Marco Slot a6d40b8bc5 Merge pull request #1452 from citusdata/citus_create_restore_point
Add citus_create_restore_point for taking distributed snapshots
2017-08-11 08:22:22 +02:00