external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,458 Commits
637 Branches
162 Tags
96 MiB
Commit Graph

4458 Commits (hack-local-execution)

Author SHA1 Message Date
Sait Talha Nisanci 4b6611460a Support foreign table joins as well 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 7e9204eba9 Update vars in quals while wrapping RTE to subquery 
When we wrap an RTE to subquery we are updating the variables varno's as
1, however we should also update the varno's of vars in quals.

Also some other small code quality improvements are done.
 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 0689f2ac1a Recursively plan distributed tables only if all have unique filters 
The previous algorithm was not consistent and it could convert different
RTEs based on the table orders in the query. Now we convert local tables
if there is a distributed table which doesn't have a unique index. So if
there are 4 tables, local1, local2, dist1, dist2_with_pkey then we will
convert local1 and local2 in `auto` mode. Converting a distributed table
is not that logical because as there is a distributed table without a
unique index, we will need to convert the local tables anyway. So
converting the distributed table with pkey is redundant.
 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci a008fc611c Support materialized view joins as well 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 5f46abffd9 Update check multi tests 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci ff4f3b2f3c Use PlannerRestrictionContext instead of RecursivePlannerContext 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 3fe3c55023 Use ShouldConvertLocalTableJoinsToSubqueries 
Remove FillLocalAndDistributedRTECandidates and use
ShouldConvertLocalTableJoinsToSubqueries, which simplifies things as we
rely on a single function to decide whether we should continue
converting RTE to subquery.
 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci eebcd995b3 Add some more tests 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 5693cabc41 Not convert an already routable plannable query 
We should not recursively plan an already routable plannable query. An
example of this is (SELECT * FROM local JOIN (SELECT * FROM dist) d1
USING(a));

So we let the recursive planner do all of its work and at the end we
convert the final query to to handle unsupported joins. While doing each
conversion, we check if it is router plannable, if so we stop.

Only consider range table entries that are in jointree

If a range table is not in jointree then there is no point in
considering that because we are trying to convert range table entries to
subqueries for join use case.
 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 2ff65f3630 Enable partitioned distributed tables in local-dist table joins 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci 44953579cf Enable citus-local distributed table joins 
Check equality in quals

We want to recursively plan distributed tables only if they have an
equality filter on a unique column. So '>' and '<' operators will not
trigger recursive planning of distributed tables in local-distributed
table joins.

Recursively plan distributed table only if the filter is constant

If the filter is not a constant then the join might return multiple rows
and there is a chance that the distributed table will return huge data.
Hence if the filter is not constant we choose to recursively plan the
local table.
 2020-12-15 18:17:10 +03:00
Sait Talha Nisanci f3d55448b3 Choose distributed table if it has a unique index in filter 
When doing local-distributed table joins we convert one of them to
subquery. The current policy is that we convert distributed tables to
subquery if it has a unique index on a column that has unique
index(primary key also has a unique index).
 2020-12-15 18:17:10 +03:00
Onder Kalaci f0aef67ed2 Update existing regression tests 2020-12-15 18:17:10 +03:00
Onder Kalaci 3f4952cc2b Pushdown projections when relations are recursively planned 
This is important to limit the data transfer size.
 2020-12-15 18:17:10 +03:00
Onder Kalaci 945193555b add basic regression tests 2020-12-15 18:17:10 +03:00
Onder Kalaci 594e001f3b Add filter pushdown regression tests 
Also handle WHERE false
 2020-12-15 18:17:10 +03:00
Onder Kalaci 82a4830c7d Adjust the existing regression tests 2020-12-15 18:17:10 +03:00
Onder Kalaci 7a4d6b2984 Handle modifications as well 2020-12-15 18:17:10 +03:00
Onder Kalaci 8f8390ed6e Recursively plan local table joins 
The logical planner cannot handle joins between local and distributed table.
Instead, we can recursively plan one side of the join and let the logical
planner handle the rest.

Our algorithm is a little smart, trying not to recursively plan distributed
tables, but favors local tables.
 2020-12-15 18:17:10 +03:00
Onder Kalaci 7cc25c9125 Add ability to fetch the restrictions per relation 
With this commit, we add the ability to add restrictions
per relation. We simply rely on the restrictions that Postgres
keeps per relation.
 2020-12-15 18:17:10 +03:00
Marco Slot 100e5d3196 Address review feedback 2020-12-15 15:23:38 +01:00
Marco Slot 23dccd8941 Add some new tests for complex correlated subqueries in WHERE 2020-12-15 14:17:16 +01:00
Marco Slot 707a6554b1 Support co-located/recurring correlated subqueries 2020-12-15 14:17:16 +01:00
Onur Tirtir 0eb5701658
Not consider single shard hash dist. tables as replicated (#4413) 2020-12-15 14:33:01 +03:00
Marco Slot cc04fce10f
Merge pull request #4360 from citusdata/marcocitus/sublinks 2020-12-13 23:46:40 +01:00
Marco Slot 4985bda3bd
Merge pull request #4384 from citusdata/marcocitus/fix/citus-tables-failure 
Harden citus_tables against node failure
 2020-12-13 23:17:15 +01:00
Marco Slot f2538a456f Support co-located/recurring sublinks in the target list 2020-12-13 15:45:24 +01:00
Marco Slot 8e8adcd92a Harden citus_tables against node failure 2020-12-13 15:10:40 +01:00
Hadi Moshayedi e24e7985f7
Merge pull request #4407 from citusdata/cstore_analyze 
Columnar: Fix ANALYZE for large number of rows.
 2020-12-10 17:32:42 -08:00
Hadi Moshayedi 4dd22cc4e4 Columnar: Fix ANALYZE for large number of rows. 2020-12-10 09:52:33 -08:00
Hadi Moshayedi df1ff60754
Merge pull request #4403 from citusdata/cstore_default_compression 
Columnar: set default compression as zstd if available
 2020-12-09 23:36:08 -08:00
Hadi Moshayedi b3dac5e9d1 Columnar: set default compression as zstd if available 2020-12-09 14:32:08 -08:00
Hadi Moshayedi 4985dcbafe
Merge pull request #4399 from citusdata/cstore_compression_level 
Columnar: Make compression level configurable
 2020-12-09 08:57:50 -08:00
Hadi Moshayedi 4668fe51a6 Columnar: Make compression level configurable 2020-12-09 08:48:50 -08:00
Hadi Moshayedi 4501310b4c
Merge pull request #4388 from citusdata/cstore_zstd 
Add zstd compression to columnar
 2020-12-09 08:37:30 -08:00
Hadi Moshayedi 9f559b37d0
Merge pull request #4337 from citusdata/cstore_lz4 
Add LZ4 compression to columnar
 2020-12-09 08:37:10 -08:00
Hadi Moshayedi f5a4a4bc74 Columnar: Support zstd compression 2020-12-09 08:30:55 -08:00
Hadi Moshayedi 3f81ee26fd Columnar: Support LZ4 compression 2020-12-09 08:29:07 -08:00
jeff-davis 260a02180b
Add tests for unsupported columnar storage features (#4397) 
Add negative tests:
 * Deletes
 * Sample scan
 * Special columns
 * Tuple locks
 * Indexes
 2020-12-09 00:08:45 -08:00
jeff-davis 776c165843
Merge pull request #4396 from citusdata/rename2 
Rename cstore->columnar in SQL objects and errors.
 2020-12-07 15:43:13 -08:00
Jeff Davis c91e5b052b more test fixups 2020-12-07 13:43:27 -08:00
Jeff Davis 7169ba21c4 more test fixes 2020-12-07 13:36:46 -08:00
Jeff Davis e26fdeb706 fixup tests some more 2020-12-07 13:22:16 -08:00
Jeff Davis 5b3c32eb38 fixup tests 2020-12-07 13:18:22 -08:00
Jeff Davis 068af7f38e fixup upgrade tests 2020-12-07 13:11:51 -08:00
Jeff Davis 3758e83850 Rename cstore->columnar in SQL objects and errors. 2020-12-07 13:01:53 -08:00
jeff-davis dee753ef05
Merge pull request #4394 from citusdata/test-update 
Tests for UPDATE and error message improvement.
 2020-12-07 12:10:13 -08:00
Jeff Davis ad919ff220 Tests for UPDATE and error message improvement. 
UPDATEs on partitioned tables that affect only row partitions should
succeed, the rest should fail.

Also rename CStoreScan to ColumnarScan to make the error message more
relevant.
 2020-12-07 11:25:30 -08:00
Ahmet Gedemenli 45ac491885
Merge pull request #4390 from citusdata/fix-transaction-name-length-calculation 
Fix transaction name length calculation
 2020-12-07 12:56:13 +03:00
Ahmet Gedemenli 7577821920 Fix transaction name length calculation 2020-12-07 12:34:15 +03:00