Commit Graph

35 Commits (c088f85b7306a3fcf2251c2cc9060d3948d7ade4)

Author SHA1 Message Date
Brian Cloutier c088f85b73 Fix crash in multi_copy test 2018-04-11 17:34:33 -07:00
Murat Tuncer 224b0a8c14 Replace poll with select/poll
Windows does not have poll(), so fall back to select()
2018-03-21 20:05:00 -07:00
Onder Kalaci 7dc9589b56 Handle failures during I/O
This commit checks the connection status right after any IO happens
on the socket.

This is necessary since before this commit we didn't pass any information
to the higher level functions whether we're done with the connection
(e.g., no IO required anymore) or an errors happened during the IO.
2018-03-02 08:33:53 +02:00
Onder Kalaci da0048e0b7 ForgetResults() becomes a wrapper for ClearResults()
ClearResults() is able to handle failures properly by
checking the result status. So, relying on it makes
error handling more generic in Citus.
2018-03-02 08:33:53 +02:00
Marco Slot 6e79a34c97 Do not check for cancellation in ClearResultsIfReady 2018-02-12 16:45:02 +01:00
Marco Slot 6051aae56e Handle errors that are discovered during abort 2018-02-12 16:45:02 +01:00
Brian Cloutier 15511f6ba1 Dynamically allocate connection metadata in WaitForAllConnections 2018-02-01 10:30:41 -08:00
Brian Cloutier a2ed45e206 Remove variable length arrays
VLAs aren't supported by Visual Studio.

- Remove all existing instances of VLAs.
- Add a flag, -Werror=vla, which makes gcc refuse to compile if we add
  VLAs in the future.
2018-02-01 10:30:41 -08:00
Hadi Moshayedi 5d7c52ffa6
Don't return in PG_TRY() block when cancellations happen in WaitForConnections(). (#1923)
We shouldn't return in middle of a PG_TRY() block because if we do, we won't reset PG_exception_stack, and later when a re-throw tries to jump to the jump-point which was active in this PG_TRY() block, it seg-faults.

We used to return in middle of PG_TRY() block in WaitForConnections() where we checked for cancellations. Whenever cancellations were caught here, Citus crashed. And example was reported by @onderkalaci at #1903.
2018-01-03 09:54:03 -05:00
Marco Slot 8f69973411 Fix cancellation issues in the real-time executor (#1905) 2018-01-01 23:10:29 -05:00
Marco Slot a64f0060ba Reduce the frequency of FinishConnectionIO calls during COPY (#1864) 2017-12-14 13:21:59 -05:00
metdos c83edc36b5 Check connection status before using it 2017-11-06 14:53:35 +02:00
Marco Slot 2bb46bb5ee Reset connectionReady flag after moving a connection in WaitForAllConnections 2017-10-31 12:06:53 +01:00
Marco Slot e6e6897499 Defer initial PQflush to main loop in WaitForAllConnections 2017-10-31 12:06:53 +01:00
Marco Slot d6dadb1b25 Use correct index for ModifyWaitEvent in WaitForAllConnections 2017-10-31 12:06:53 +01:00
Murat Tuncer 4676c4f7a5 Prevent crash when remote transaction start fails (#1662)
We sent multiple commands to worker when starting a transaction.
Previously we only checked the result of the first command that
is transaction 'BEGIN' which always succeeds. Any failure on
following commands were not checked.

With this commit, we make sure all command results are checked.
If there is any error we report the first error found.
2017-09-26 17:25:46 -07:00
Onder Kalaci 4782f9f98a Properly copy and trim the error messages that come from pg_conn
When a NULL connection is provided to PQerrorMessage(), the
returned error message is a static text. Modifying that static
text, which doesn't necessarly be in a writeable memory, is
dangreous and might cause a segfault.
2017-09-22 19:43:09 +03:00
Marco Slot d1befa4df9 Wait for I/O to finish after PQputCopyData 2017-09-12 16:18:42 -07:00
Marco Slot fca986f214 Add API for waiting for multiple connections 2017-08-11 00:03:06 +02:00
Onder Kalaci 3369f3486f Introduce distributed transaction ids
This commit adds distributed transaction id infrastructure in
the scope of distributed deadlock detection.

In general, the distributed transaction id consists of a tuple
in the form of: `(databaseId, initiatorNodeIdentifier, transactionId,
timestamp)`.

Briefly, we add a shared memory block on each node, which holds some
information per backend (i.e., an array `BackendData backends[MaxBackends]`).
Later, on each coordinated transaction, Citus sends
`SELECT assign_distributed_transaction_id()` right after `BEGIN`.
For that backend on the worker, the distributed transaction id is set to
the values assigned via the function call.

The aim of the above is to correlate the transactions on the coordinator
to the transactions on the worker nodes.
2017-07-18 15:01:42 +03:00
Andres Freund be8677f926 Add NonblockingForgetResults().
This is very similar to ForgetResults() except that no network IO is
performed. Primarily useful in error handling cases.
2017-07-04 14:46:03 -07:00
Andres Freund 75a7ddea0d Always use connections in non-blocking mode.
Now that there's no blocking libpq callers left, default to using
non-blocking mode in connection_management.c.  This has two
advantages:
1) Blockiness doesn't have to frequently be reset, simplifying code
2) Prevents accidental use of blocking libpq functions, since they'll
   frequently return 'need IO'
2017-07-04 14:46:03 -07:00
Andres Freund c674bc8640 Add interrupt aware PQputCopy{End,Data} wrappers. 2017-07-04 12:38:52 -07:00
Andres Freund b7f9679ccc Move interrupt handling code from GetRemoteCommandResult to FinishConnectionIO.
Nearby commits will add additional interrupt handling functions, this
way we don't have to duplicate the code.
2017-07-04 12:38:52 -07:00
Jason Petersen 2204da19f0 Support PostgreSQL 10 (#1379)
Adds support for PostgreSQL 10 by copying in the requisite ruleutils
and updating all API usages to conform with changes in PostgreSQL 10.
Most changes are fairly minor but they are numerous. One particular
obstacle was the change in \d behavior in PostgreSQL 10's psql; I had
to add SQL implementations (views, mostly) to mimic the pre-10 output.
2017-06-26 02:35:46 -06:00
Metin Doslu b6659bec22 Send explain queries with savepoints
With this commit, we started to send explain queries within a savepoint. After
running explain query, we rollback to savepoint. This saves us from side effects
of EXPLAIN ANALYZE on DML queries.
2017-04-28 12:13:48 -07:00
Andres Freund fa5b8fb39f Fix SendRemoteCommandParams() handling of a NULL MultiConnection->pgConn. (#1271)
Previously we'd segfault in PQisnonblocking() which, contrary to other
libpq calls, doesn't handle a NULL PQconn (because there'd be no
appropriate return value for that).

cr: @jasonmp85
2017-03-03 12:02:15 -07:00
Brian Cloutier 67ee357d7f Port WorkerShardStats to new connection API
Part of the work in citusdata/citus#1101, this is a pretty direct port
over to the new functions and shouldn't result in any behavior changes.
2017-01-17 17:04:37 +02:00
Brian Cloutier b1b2b4fadf Create ExecuteOptionalRemoteCommand
A small refactor which pulls some code out of `RecoverWorkerTransactions`
and into `remote_commands.c`. This code block currently only occurs in
`RecoverWorkerTransactions` but will be useful to other functions
shortly.

Unfortunately we couldn't call it `ExecuteRemoteCommand`, that name was
already taken.
2017-01-17 17:04:37 +02:00
Marco Slot ef326b202a PQclear in ReportResultError to prevent memory leaks 2017-01-10 02:51:39 +01:00
Marco Slot 92c7567008 Convert worker_transactions to new connection API 2016-12-23 16:14:29 +01:00
Marco Slot 00d55ad957 Add a wrapper for PQsendQuery 2016-12-23 16:14:29 +01:00
Marco Slot 87c62d598e Connectionapify SendCommandListToWorkerInSingleTransaction 2016-12-23 16:14:29 +01:00
Andres Freund 6eeb43af15 Add PQgetResult() wrapper handling interrupts.
This makes it possible to implement cancelling queries blocked on
communication with remote nodes.
2016-12-12 15:18:12 -08:00
Andres Freund 3505d431cd Add initial helpers to make interactions with MultiConnection et al. easier.
This includes basic infrastructure for logging of commands sent to
remote/worker nodes. Note that this has no effect as of yet, since no
callers are converted to the new infrastructure.
2016-12-07 11:44:24 -08:00