Commit Graph

822 Commits (e3c763c3f75fb65366b8ca421e944611ee3a39b5)

Author SHA1 Message Date
Marco Slot 5714be0da5 Expose the column_to_column_name UDF to make partkey in pg_dist_partition human-readable 2016-12-14 10:46:33 +01:00
Eren Başak 9d4e586457 Merge pull request #997 from citusdata/add_sync_metadata_to_node
Add sync_metadata_to_node UDF
2016-12-14 10:06:17 +02:00
Eren Basak afbb5ffb31 Add stop_metadata_sync_to_node UDF 2016-12-14 10:53:12 +03:00
Eren Basak b94647c3bc Propagate CREATE SCHEMA commands with the correct AUTHORIZATION clause in start_metadata_sync_to_node 2016-12-14 10:53:12 +03:00
Eren Basak fb08093b00 Make start_metadata_sync_to_node UDF to propagate foreign-key constraints 2016-12-14 10:53:12 +03:00
Eren Basak 5e96e4f60e Make truncate triggers propagated on start_metadata_sync_to_node call 2016-12-14 10:53:10 +03:00
Eren Basak 4fd086f0af Prevent Transactions in start_metadata_sync_to_node 2016-12-13 10:48:03 +03:00
Eren Basak c154a91621 Add Regression Tests For start_metadata_sync_to_node 2016-12-13 10:48:03 +03:00
Eren Basak 9eff968d1f Add start_metadata_sync_to_node UDF
This change adds `start_metadata_sync_to_node` UDF which copies the metadata about nodes and MX tables
from master to the specified worker, sets its local group ID and marks its hasmetadata to true to
allow it receive future DDL changes.
2016-12-13 10:48:03 +03:00
Andres Freund 21effef8b5 Merge pull request #835 from citusdata/valgrind-support
Allow to run regression tests under valgrind
2016-12-12 16:34:58 -08:00
Andres Freund 56aaa25cfa Add support for running regression tests under valgrind.
Note that this only provides infrastructure for running tests under
valgrind - there's some spurious failures due to timeouts.
2016-12-12 15:42:11 -08:00
Andres Freund dd149c3e24 Merge pull request #1020 from citusdata/feature/transaction-management
Centralized Transaction Management Infrastructure
2016-12-12 15:27:27 -08:00
Andres Freund 80b34a5d6b Integrate router executor into transaction management framework.
One less place managing remote transactions. It also makes it fairly
easy to use 2PC for certain modifications (e.g. reference tables). Just
issue a CoordinatedTransactionUse2PC(). If every placement failure
should cause the whole transaction to abort, additionally mark the
relevant transactions as critical.
2016-12-12 15:18:12 -08:00
Andres Freund fa5e202403 Convert multi_shard_transaction.[ch] to new framework. 2016-12-12 15:18:12 -08:00
Andres Freund fc298ec095 Coordinated remote transaction management. 2016-12-12 15:18:12 -08:00
Andres Freund 6eeb43af15 Add PQgetResult() wrapper handling interrupts.
This makes it possible to implement cancelling queries blocked on
communication with remote nodes.
2016-12-12 15:18:12 -08:00
Andres Freund 5a00de6c62 Merge pull request #1017 from citusdata/vanillatests
Add support for running postgres tests against a database with citus loaded
2016-12-12 14:34:20 -08:00
Andres Freund b65814280f Add support for running postgres tests against a database with citus loaded.
Not that this is not with the citus *extension* loaded, just the shared
library. The former runs (by adding --use-existing to the flags) but
has a bunch of trivial test differences.
2016-12-12 14:25:06 -08:00
Jason Petersen a426c50a8c Merge pull request #1035 from citusdata/tweak_coverage
Tweak coverage options

cr: @jasonmp85
2016-12-09 14:19:52 -07:00
Jason Petersen 2ca9c78540
Bump target to 87.5%
With the ignore rules in place, we can increase the target.
2016-12-09 14:06:35 -07:00
Jason Petersen 1e4f5e1a95
Format for readability 2016-12-09 13:33:58 -07:00
Jason Petersen 145537a5d2
Add ignore rules 2016-12-09 13:03:23 -07:00
Jason Petersen 14f4dfb242
Make patch target 75%
If a patch has very bad coverage, reject it.
2016-12-09 13:03:23 -07:00
Jason Petersen 5bd07b253a
Be explicit about coverage project requirements
If a change drops coverage (absolutely) beneath 80% or (relatively)
reduces coverage by more than 0.5%, give it a bad status.
2016-12-09 13:03:23 -07:00
Jason Petersen 26ae8704d6
Add default Codecov configuration
Will change shortly.
2016-12-09 13:03:23 -07:00
Andres Freund 4bea1f621a Merge pull request #1022 from citusdata/feature/2PCBUMPISM
Make prepared transactions available if not configured.
2016-12-08 20:04:00 -08:00
Andres Freund 7434fcc6df Make prepared transactions available if not configured. 2016-12-08 19:57:22 -08:00
Burak Yücesoy 62b12f413c Merge pull request #876 from citusdata/foreign_key_push_down_for_alter_table
Add Foreign Key Support to ALTER TABLE commands
2016-12-08 14:11:01 +02:00
Burak Yucesoy 8d7cd4d746 Add Foreign Key Support to ALTER TABLE commands
With this PR, we add foreign key support to ALTER TABLE commands. For now,
we only support foreign constraint creation via ALTER TABLE query, if it
is only subcommand in ALTER TABLE subcommand list.

We also only allow foreign key creation if replication factor is 1.
2016-12-08 15:03:25 +02:00
Andres Freund f7d9074aa3 Merge pull request #863 from citusdata/feature/connection-lifecycle
Connection Lifecycle Management Infrastructure
2016-12-07 11:51:02 -08:00
Andres Freund 2374905c89 Move multi_client_executor.[ch] ontop of connection_management.[ch].
That way connections can be automatically closed after errors and such,
and the connection management infrastructure gets wider testing.  It
also fixes a few issues around connection string building.
2016-12-07 11:44:24 -08:00
Andres Freund a77cf36778 Use connection_management.c from within connection_cache.c.
This is a temporary step towards removing connection_cache.c.
2016-12-07 11:44:24 -08:00
Andres Freund 3505d431cd Add initial helpers to make interactions with MultiConnection et al. easier.
This includes basic infrastructure for logging of commands sent to
remote/worker nodes. Note that this has no effect as of yet, since no
callers are converted to the new infrastructure.
2016-12-07 11:44:24 -08:00
Andres Freund 3223b3c92d Centralized Connection Lifetime Management.
Connections are tracked and released by integrating into postgres'
transaction handling. That allows to to use connections without having
to resort to having to disable interrupts or using PG_TRY/CATCH blocks
to avoid leaking connections.

This is intended to eventually replace multi_client_executor.c and
connection_cache.c, and to provide the basis of a centralized
transaction management.

The newly introduced transaction hook should, in the future, be the only
one in citus, to allow for proper ordering between operations.  For now
this central handler is responsible for releasing connections and
resetting XactModificationLevel after a transaction.
2016-12-07 11:43:18 -08:00
Andres Freund 883af02b54 Add some basic helpers to make use of dynahash hashtables easier. 2016-12-06 14:15:36 -08:00
Jason Petersen 4347daaf4c Merge pull request #994 from citusdata/enable-coverage-testing
Add coverall support for continuous code coverage testing

cr: @jasonmp85
2016-12-06 11:42:48 -07:00
Jason Petersen ac6494a0c3
Add comment for otherwise opaque secure value 2016-12-06 11:30:22 -07:00
Brian Cloutier 29d2b0c49f
Enable instrumentation of coverage
Adds an --enable-coverage configure option which provides the necessary
flags for coverage instrumentation. A new tools branch uses this flag
during all builds. Coverage reports are uploaded to codecov.io, where
they are publicly visible.
2016-12-06 11:30:22 -07:00
Marco Slot f2d151cfeb Merge pull request #1006 from citusdata/bugfix/placement_id_readfunc
Use READ_UINT64_FIELD for placement ID in ReadShardPlacement
2016-12-05 23:08:27 +01:00
Marco Slot 3d09a2e5c2 Use READ_UINT64_FIELD for placement ID in ReadShardPlacement 2016-12-05 17:22:23 +01:00
Murat Tuncer 51f2de276e Merge pull request #995 from citusdata/fix-870-non-relational-filter
Add new tests for non-relational filters in queries
2016-12-05 13:35:46 +02:00
Murat Tuncer 131ed8ca1f Add new tests for non-relational filters in queries 2016-12-05 14:27:36 +03:00
Marco Slot ab982c3ecb Merge pull request #911 from citusdata/bugfix/take_metadata_lock
Take shard metadata lock in several UDFs
2016-12-02 16:34:44 +01:00
Marco Slot 172bb457e6 Take shard metadata lock in master_append_table_to_shard 2016-12-02 15:56:30 +01:00
Eren Başak 8013437fc3 Merge pull request #996 from citusdata/sync_pg_dist_node
Propagate node add/remove to the nodes with hasmetadata=true
2016-12-02 13:50:52 +02:00
Eren Basak fb88b167a7 Propagate node add/remove to the nodes with hasmetadata=true
This change propagates the changes done by `master_add_node` and `master_remove_node`
to the workers that contain metadata.
2016-12-02 14:43:32 +03:00
Brian Cloutier a4096c9f45 Remove dead code: ResponsiveWorkerNodeList 2016-12-02 13:14:11 +03:00
Andres Freund 0a4889d0af Use system psql if available, to fix travis build errors.
On some systems a new libpq is available than what we're compiling
against, but until now we used psql in the version we're compiling
against.  That' a problem, because (quoting Jason):

  With 9.6, libpq's default handling of CONTEXT changed: it is hidden
  unless the level is ERROR or higher. We addressed this ourselves using
  the SHOW_CONTEXT variable (by setting "always" in pg_regress_multi): in
  9.5, this is ignored (and unneeded), in 9.6, it ensures old behavior is
  preserved.

For 9.6 we'd already worked around the problem by specifying that
context should always be shown, but < 9.6 psql doesn't know how to do
that.

As there's no csql anymore, which strictly tied us to a specific version
of psql/csql, we can now just use the system's psql if available. We
still fall back to the psql of the installation we're compiling against,
if there's no other psql in PATH.
2016-12-01 15:58:23 -08:00
Önder Kalacı 89352ae83f Merge pull request #988 from citusdata/fix_constant_select
Bugfix for deparsing INSERT..SELECT queries which involve constant va…
2016-12-01 10:48:01 +02:00
Onder Kalaci df974e15b8 Bugfix for deparsing INSERT..SELECT queries which involve constant values
This commit fixes a bug when the SELECT target list includes a constant
value.

Previous behaviour of target list re-ordering:
  * Iterate over the INSERT target list
    * If it includes a Var, find the corresponding SELECT entry
      and update its resno accordingly
    * If it does not include a Var (which we only considered to be
      DEFAULTs), generate a new SELECT target entry
  * If the processed target entry count in SELECT target list is less
    than the original SELECT target list (GROUP BY elements not included in
    the SELECT target entry), add them in the SELECT target list and
    update the resnos accordingly.
     * However, this step was leading to add the CONST SELECT target entries
       twice. The reason is that when CONST target list entries appear in the
       SELECT target list, the INSERT target list doesn't include a Var. Instead,
       it includes CONST as it does for DEFAULTs.

New behaviour of target list re-ordering:
  * Iterate over the INSERT target list
    * If it includes a Var, find the corresponding SELECT entry
      and update its resno accordingly
    * If it does not include a Var (which we consider to be
      DEFAULTs and CONSTs on the SELECT), generate a new SELECT
      target entry
  * If any target entry remains on the SELECT target list which are resjunk,
    (GROUP BY elements not included in the SELECT target entry), keep them
    in the SELECT target list by updating the resnos.
2016-12-01 10:41:56 +02:00