Commit Graph

3402 Commits (8deb8053389b9741a8ffd2c90e242d71d6d3280f)

Author SHA1 Message Date
Philip Dubé 70436ec279
Merge pull request #3587 from citusdata/fix-typos
Fix typos, rename isDistributedRelation to isCitusRelation
2020-03-07 14:21:03 +00:00
Philip Dubé a7cca1bcde Rename DistTableCacheEntry to CitusTableCacheEntry 2020-03-07 14:08:03 +00:00
Philip Dubé b514ab0f55 Fix typos, rename isDistributedRelation to isCitusRelation 2020-03-06 19:20:34 +00:00
Philip Dubé 00a7bc3044
Merge pull request #3586 from citusdata/rename-distributed-to-citus
Try to create clear distinction between DistributedTable vs CitusTable
2020-03-06 19:10:16 +00:00
Philip Dubé bec58000d6 Given IsDistributedTableRTE, there's ambiguity in what DistributedTable means
Elsewhere we used DistributedTable to include reference tables
Marco suggested we use CitusTable for distributed & reference tables

So renaming:
- IsDistributedTable -> IsCitusTable
- IsDistributedTableViaCatalog -> IsCitusTableViaCatalog
- DistributedTableCacheEntry -> CitusTableCacheEntry
- DistributedTableList -> CitusTableList
- isDistributedTable -> isCitusTable
- InsertSelectIntoDistributedTable -> InsertSelectIntoCitusTable
- ExtractFirstDistributedTableId -> ExtractFirstCitusTableId
2020-03-06 18:57:55 +00:00
Onur Tirtir a381074787
Update CHANGELOG for 9.0.2 (#3585)
(cherry picked from commit de6068b2c4)

Co-authored-by: Hanefi Onaldi <hanefionaldi@gmail.com>
2020-03-06 18:26:04 +03:00
Marco Slot fa29fb8c52
Merge pull request #3579 from citusdata/disable_postgres_parallelism
Disable Postgres parallelism by default in tests
2020-03-06 15:54:57 +01:00
Onur Tirtir 50e59f1a61
Update CHANGELOG for 9.2.2 (#3582)
Co-authored-by: Hanefi Onaldi <hanefionaldi@gmail.com>
2020-03-06 16:09:34 +03:00
Marco Slot 5b1d1dd413 Remove unnecessary use of max_parallel_workers_per_gather 2020-03-06 13:18:58 +01:00
Marco Slot d0fead6691 Disable Postgres parallelism by default in tests 2020-03-06 13:18:58 +01:00
Onur Tirtir c5007bc93c
Merge pull request #3563 from citusdata/refactor/local-group-id-and-fkey
Refactor around foreign key constraints and GetLocalGroupId
2020-03-05 20:32:10 +03:00
Onur Tirtir bdce9acc30 some refactor around foreign key constraints 2020-03-05 20:20:41 +03:00
Onur Tirtir 88bfd2e4b7 refactor around local group id checks
Mostyl optimizes the calls made to GetLocalGroupId and refactors
its usages
2020-03-05 20:20:41 +03:00
Onur Tirtir 1e128a6ee4 fix a potential infinite loop 2020-03-05 20:20:41 +03:00
SaitTalhaNisanci a75436a54b
refactor CoordinatedTransactionCallback (#3571) 2020-03-05 18:36:12 +03:00
Hanefi Onaldi 6e6763678c
Merge pull request #3564 from citusdata/fix-early-exits-on-subplan-pruning
Fix early exits on intermediate result pruning

There are 2 problems with our early exit strategy that this commit fixes:

1- When we decide that a subplan results are sent to all worker nodes,
we used to skip traversing the whole distributed plan, instead of
skipping only the subplan.

2- We used to consider all available nodes in the cluster (secondaries
and inactive nodes as well as active primaries) when deciding on early
exit strategy. This resulted in failures to early exit when there are
secondaries or inactive nodes.
2020-03-05 16:51:15 +03:00
Hanefi Onaldi c0ad44f975
Fix early exit bug on intermediate result pruning
There are 2 problems with our early exit strategy that this commit fixes:

1- When we decide that a subplan results are sent to all worker nodes,
we used to skip traversing the whole distributed plan, instead of
skipping only the subplan.

2- We used to consider all available nodes in the cluster (secondaries
and inactive nodes as well as active primaries) when deciding on early
exit strategy. This resulted in failures to early exit when there are
secondaries or inactive nodes.
2020-03-05 16:41:44 +03:00
Marco Slot 241c186603
Merge pull request #3553 from citusdata/refactor/begin_scan
Refactor CitusBeginScan into separate SELECT / DML paths
2020-03-05 13:06:10 +01:00
Onder Kalaci f72916875f Expand test coverage for combinations of master evalution, deferred pruning, parameters, local execution
- Router           & Remote & Requires Master Evaluation & With Param & Without Param
- Fast Path Router & Remote & Requires Master Evaluation & With Param & Without Param
2020-03-05 12:37:22 +01:00
Marco Slot dc4c0c032e Refactor CitusBeginScan into separate DML / SELECT paths 2020-03-05 12:37:22 +01:00
Nils Dijk 268ad741a9
Refactor the deparsing of a CREATE EXTENSION to prevent NULL POINTER dereferences (#3518)
DESCRIPTION: satisfy static analysis tool for a nullptr dereference

During the static analysis project on the codebase this code has been flagged as having the potential for a null pointer dereference. Funnily enough the author had already made a comment of it in the code this was not possible due to us setting the schema name before we pass in the statement. If we want to reuse this code in a later setting this comment might not always apply and we could actually run into null pointer dereference.

This patch changes a bit of the code around to first of all make sure there is no NULL pointer dereference in this code anymore.
Secondly we allow for better deparsing by setting and adhering to the `if_not_exists` flag on the statement.
And finally add support for all syntax described in the documentation of postgres (FROM was missing).
2020-03-04 16:47:07 +01:00
Önder Kalacı 9096c650f6
Merge pull request #3562 from citusdata/add_type_to_deparse
For composite types, add cast to the parameter to ease remote node detect the type
2020-03-04 13:12:05 +01:00
Marco Slot 27f23d2c89 Add some distribution column = composite type prepared statement tests 2020-03-04 05:01:43 +01:00
Onder Kalaci 087f6eb4c0 For composite types, add cast to the parameter to ease remote node detect
the type.
2020-03-04 11:27:45 +01:00
Onur Tirtir c9c6e58c53
Merge pull request #3554 from citusdata/refactor/vacuum-and-local-executor
Refactor vacuumTaskList function and local_executor.c line lengths
2020-03-02 12:04:55 +03:00
Onur Tirtir ff9c9d1808 make VacuumTaskList even with other taskList functions and some safety changes
Makees VacuumTaskList function even with other TaskList creator functions.
Also, previously we were generating per-shard vacuum command strings via
unconventional usage of StringInfo struct (setting the stringInfo->len field
manually) which could cause unexepected memory errors (that I cannot foresee now).
2020-03-02 10:25:28 +03:00
Onur Tirtir cf718ffe77 safely error out in DistributedTableCacheEntry function 2020-03-02 10:25:12 +03:00
Onur Tirtir 17d9b934c3 refactor local_executor.c lines with >78 characters 2020-02-29 15:04:34 +03:00
Philip Dubé 6dbb48c9f1
Merge pull request #3550 from citusdata/fix-generated-halfway
Fix create_distributed_table on a table using GENERATED ALWAYS AS
2020-02-28 17:56:16 +00:00
Philip Dubé 34f241af16 Fix create_distributed_table on a table using GENERATED ALWAYS AS
If the generated column does not come at the end of the column list,
columnNameList doesn't line up with the column indexes. Seek past

CREATE TABLE test_table (
    test_id int PRIMARY KEY,
    gen_n int GENERATED ALWAYS AS (1) STORED,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
SELECT create_distributed_table('test_table', 'test_id');

Would raise ERROR: cannot cast 23 to 1184
2020-02-28 09:34:26 -08:00
Philip Dubé 2fae132e45
repartition_join_execution: Don't store 64 bit integers as poin… (#3551)
Pointers are not necessarily 64bit
2020-02-28 15:06:06 +01:00
Philip Dubé 20abc4d2b5
Replace foreach with foreach_ptr/foreach_oid (#3544) 2020-02-27 16:54:49 +01:00
Philip Dubé 99589de5f9
Merge pull request #3545 from citusdata/make-implicit-cell-harder
Make bad refactors to foreach_xxx error out
2020-02-27 13:16:59 +00:00
Jelte Fennema c48f0ca7e5 Make bad refactors to foreach_xxx error out
Without this commit you could still use varCell in the body of loop.
This makes it easy for bad refactors that still use the ListCell to slip
through unnoticed, because the new ListCell will be named the same as the
one used in the old code. By renaming the ListCell to varCellDoNotUse
this will not happen.
2020-02-27 10:59:45 +01:00
Jelte Fennema 685b54b3de
Semmle: Check for NULL in some places where it might occur (#3509)
Semmle reported quite some places where we use a value that could be NULL. Most of these are not actually a real issue, but better to be on the safe side with these things and make the static analysis happy.
2020-02-27 10:45:29 +01:00
Jelte Fennema f6a89bcd12
Merge pull request #3541 from citusdata/jelte_fix
We got some errors for safestringlib builds on OSX. The fixes are as follows:

1.  Change name of memset_s to memset8_s
2.  Disable some linker flags on OSX
3.  Reorder warning flags in so -Wall does not override an ignore for clang

Also adds clean-full target to clean our compiled code and also vendored artifacts. Usually it's not needed to clean vendored artifacts once they are built correctly, so they are not cleaned during regular clean to keep full recompiles of our code faster.
2020-02-26 17:59:03 +01:00
Jelte Fennema 0cad263c82 Add new vendor README update instructions 2020-02-26 17:46:37 +01:00
Jelte Fennema eb8e099f09 Fix Makefile so that it builds safestringlib correctly on OSX 2020-02-26 17:44:44 +01:00
Jelte Fennema 8e7eaaf949 Add clean-full to also clean full builds of vendored libraries 2020-02-26 17:44:44 +01:00
Jelte Fennema 92d7a40d1d Fix safestringlib build on OSX 2020-02-26 17:44:44 +01:00
Hadi Moshayedi 8ca55c739f
Merge pull request #3543 from citusdata/MarkusSintonen-improve-shard-pruning
(MarkusSintonen) Improve shard pruning logic to understand OR-conditions
2020-02-26 07:37:26 -08:00
Hadi Moshayedi e7cce40e6e Address pykello's feedback 2020-02-26 07:17:32 -08:00
Hadi Moshayedi 1b3e58f0c3 Merge branch 'improve-shard-pruning' of https://github.com/MarkusSintonen/citus into MarkusSintonen-improve-shard-pruning 2020-02-26 07:13:33 -08:00
SaitTalhaNisanci 82d22b34fe
create temp schemas in parallel (#3540) 2020-02-26 16:20:08 +03:00
SaitTalhaNisanci d94c3fd43d
send repartition cleanup jobs in parallel to all workers (#3485)
* send repartition cleanup jobs in parallel to all workers

* add review items
2020-02-26 13:44:06 +03:00
Marco Slot 1b6020e2d6
Merge pull request #3539 from citusdata/unlogged_merge_tables
Make merge tables during re-partitioning unlogged
2020-02-26 10:55:15 +01:00
Marco Slot c7f123947e Make merge tables during re-partitioning unlogged 2020-02-26 10:46:07 +01:00
Jelte Fennema 5d601bb45a
Merge pull request #3465 from citusdata/safestringlib
Use safestringlib for safe buffer interaction
2020-02-25 16:33:44 +01:00
Jelte Fennema 62bf571ced Make SafeSnprintf work on PG11 2020-02-25 15:39:27 +01:00
Jelte Fennema 7d24cebc80 Add pg11 snprintf file to repo for use in pg11 when it's not compiled 2020-02-25 15:39:27 +01:00