external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,356 Commits
636 Branches
162 Tags
98 MiB
Commit Graph

4356 Commits (f7c70f9a63f53efb988eef0389946b1c38248ff3)

Author SHA1 Message Date
Jeff Davis a3b513167c disable a few tests 2020-09-16 11:10:40 -07:00
Jeff Davis c49acc948a more test fixes........ 2020-09-16 11:10:40 -07:00
Jeff Davis fd6b4aeba2 more tests... 2020-09-16 11:10:39 -07:00
Jeff Davis 7ba75fc2a6 more tests pass 2020-09-16 11:10:39 -07:00
Jeff Davis 83f2d4aef2 more fixes 2020-09-16 11:10:39 -07:00
Jeff Davis 18f6829621 more fixes 2020-09-16 11:10:39 -07:00
Jeff Davis a57b9004a4 tests WIP 2020-09-16 11:10:39 -07:00
Jeff Davis f886fb33e5 add AM tests 2020-09-16 11:10:39 -07:00
Jeff Davis aa422f2da0 fixup rebase 2020-09-16 11:10:39 -07:00
Jeff Davis b06f48a2a7 tableAM updates 2020-09-16 11:10:39 -07:00
Jeff Davis b6ca8fcd70 extension control 2020-09-16 11:10:39 -07:00
Jeff Davis 48e9c17b50 stubs for table access method 2020-09-16 11:10:39 -07:00
jeff-davis 30b78d6f54 Merge pull request #8 from citusdata/circleci-project-setup 
Circleci project setup
 2020-09-16 09:42:14 -07:00
Nils Dijk 1e93e15a8d fix indentation via citus_indent 2020-09-16 15:22:09 +02:00
Nils Dijk 20a8bca426 add integration files for circle ci 
This is based on the circle ci integration we have
for citus, albeit highly simplified.
 2020-09-16 15:21:42 +02:00
Nils Dijk 09208986ba remove travis 2020-09-16 15:21:42 +02:00
Jeff Davis fe7ab6df84 Rename tests to be FDW-specific. 2020-09-15 12:51:15 -07:00
Jeff Davis f7f59933f8 fix v11 tests 2020-09-15 12:48:44 -07:00
Hadi Moshayedi d69bff7621 Use schema config in control file 2020-09-15 10:06:11 -07:00
Onur Tirtir 4118560b75
Prevent citus local table creation from a catalog table (#4158) 2020-09-15 14:30:48 +03:00
Nils Dijk 00cb58135d Merge pull request #7 from citusdata/add-wal 
Add wal support
 2020-09-15 12:41:41 +02:00
Nils Dijk a94bbcc7ef write wal entries when writing to the buffers 2020-09-15 12:38:50 +02:00
Hadi Moshayedi 139da88ad9 Remove some unnecessary code & fix compiler warnings 2020-09-14 15:08:50 -07:00
Hadi Moshayedi c1cf3fe6e7 Merge pull request #5 from citusdata/skiplist_to_metadata_tables 
Move skipnodes to metadata tables
 2020-09-14 15:00:29 -07:00
Hadi Moshayedi 2737686fd0 Move skipnodes to metadata tables 2020-09-14 14:57:13 -07:00
jeff-davis c570932712 Merge pull request #6 from citusdata/smgr 
Smgr
 2020-09-14 13:52:25 -07:00
Hadi Moshayedi fb110446be Fix compilation in pg 11 2020-09-14 13:13:36 -07:00
Önder Kalacı e7079d1384
Add orderbys to some tests (#4162) 2020-09-14 16:59:22 +02:00
Jeff Davis 573555747f address review comments 2020-09-11 16:28:57 -07:00
jeff-davis b8b5d3aeee Merge pull request #4 from citusdata/fdw-relfilenode 
create relfilenode for FDW
 2020-09-11 16:16:55 -07:00
Jeff Davis dee408248c Replace file access with Smgr 2020-09-11 16:14:36 -07:00
Jeff Davis a2f7eadeb9 lock while initializing relfilenode 2020-09-11 16:02:00 -07:00
Jeff Davis b18c9c8060 drop storage for DROP command 2020-09-11 15:04:46 -07:00
Jeff Davis e9045227cd create relfilenode for FDW 2020-09-11 12:48:00 -07:00
Marco Slot 94736ce78d
Merge pull request #3938 from citusdata/fix/extension-dist-tables 2020-09-11 12:24:35 +02:00
Onur Tirtir 9a56c22917
Add udf tests with citus local tables (#4154) 2020-09-11 12:36:53 +03:00
Hadi Moshayedi 407892a9dd Merge pull request #2 from citusdata/table_footer_to_metadata_tables 
Move table footer to metadata tables
 2020-09-09 22:27:23 -07:00
Marco Slot b82f6ee163 Add tests for distributing catalog tables 2020-09-10 04:46:11 +02:00
Marco Slot bd12555b16 Fix distributing tables owned by extensions 2020-09-10 04:46:11 +02:00
Hadi Moshayedi 0d4e249c97 Reuse the same state for multiple inserts 2020-09-09 14:17:30 -07:00
Hadi Moshayedi 35a52a6fe1 Use cstore namespace instead of pg_catalog. 2020-09-09 11:04:27 -07:00
Onur Tirtir 5e5ba46793
Merge pull request #4143 from citusdata/single-placement-table/master-cache-entry-rebased 
DESCRIPTION: Introduce citus local tables

The commits in this pr are merged from other sub-pr's:

* community/#3852: Brings lazy&fast table creation logic for create_citus_local_table udf
* community/#3995: Brings extended utility command support for citus local tables
* community/#4133: Brings changes in planner and in several places to integrate citus local tables into our distributed execution logic

We are introducing citus local tables, which a new table type to citus.

To be able to create a citus local table, first we need to add coordinator as a worker
node.
Then, we can create a citus local table via SELECT create_citus_local_table(<tableName>).

Calling this udf from coordinator will actually create a single-shard table whose shard
is on the coordinator.

Also, from the citus metadata perspective, for citus local tables:
* partitionMethod is set to DISTRIBUTE_BY_NONE (like reference tables) and
* replicationModel is set to the current value of citus.replication_model, which
  already can't be equal to REPLICATION_MODEL_2PC, which is only used for reference
  tables internally.

Note that currently we support creating citus local tables only from postgres tables
living in the coordinator.
That means, it is not allowed to execute this udf from worker nodes or it is not allowed
to move shard of a citus local table to any other nodes.

Also, run-time complexity of calling create_citus_local_table udf does not depend
on the size of the relation, that means, creating citus local tables is actually a
non-blocking operation.
This is because, instead of copying the data to a new shard, this udf just does the
following:

* convert input postgres table to the single-shard of the citus local table by suffixing
  the shardId to it's name, constraints, indexes and triggers etc.,
* create a shell table for citus local table in coordinator and in mx-worker nodes when
  metadata sycn is enabled.
* create necessary objects on shell table.

Here, we should also note we can execute queries/dml's from mx worker nodes
as citus local tables are already first class citus tables.

Even more, we brought trigger support for citus local tables.
That means, we can define triggers on citus local tables so that users can define trigger
objects to perform execution of custom functions that might even modify other citus tables
and other postgres tables.

Other than trigger support, citus local tables can also be involved in foreign key relationships
with reference tables.
Here the only restriction is, foreign keys from reference tables to citus local tables cannot
have behaviors other than RESTRICT & NO ACTION behavior.
Other than that, foreign keys between citus local tables and reference tables just work fine.

All in all, citus local tables are actually just local tables living in the coordinator, but natively
accessible from other nodes like other first class citus tables and this enables us to set foreign
keys constraints between very big coordinator tables and reference tables without having to
do any data replication to worker nodes for local tables.
 2020-09-09 13:02:42 +03:00
Onur Tirtir 3a73fba810 Apply planner changes for citus local tables 2020-09-09 11:51:18 +03:00
Onur Tirtir 0b1cc118a9 Adapt other cache entry changes for citus local tables 2020-09-09 11:50:55 +03:00
Onur Tirtir a58a4395ab Extend citus local table utility command support 
This commit brings following features:

Foreign key support from citus local tables to reference tables
* Foreign key support from reference tables to citus local tables
  (only with RESTRICT & NO ACTION behavior)
* ALTER TABLE ENABLE/DISABLE trigger command support
* CREATE/DROP/ALTER trigger command support

and disallows:
* ALTER TABLE ATTACH/DETACH PARTITION commands
* CREATE TABLE <postgres table> ATTACH PARTITION <citus local table>
  commands
* Foreign keys from postgres tables to citus local tables
  (the other way was already disallowed)

for citus local tables.
 2020-09-09 11:50:55 +03:00
Onur Tirtir 17cc810372 Implement "citus local table" creation logic 2020-09-09 11:50:48 +03:00
Hadi Moshayedi 10fd94a9e3 Address feedback 2020-09-08 19:05:07 -07:00
Hadi Moshayedi 9e247cdf40 Move table footer to metadata tables 2020-09-07 21:53:28 -07:00
Hadi Moshayedi 85a51fb2ef Merge pull request #3 from citusdata/add_reindent 
Add 'make reindent'
 2020-09-07 15:51:09 -07:00
Hadi Moshayedi b74de68ce3 Add 'make reindent' 2020-09-07 15:48:23 -07:00