external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,342 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

5342 Commits (78c9802d9cc6c5fe7d4593f1fdd0706263d1143c)

Author SHA1 Message Date
Nils Dijk 78c9802d9c
basic geo partitioning planning 2021-11-09 14:30:27 +01:00
Nils Dijk 03d2dc9e71
add sql script to initialize a geo distributed table 2021-11-09 14:30:26 +01:00
Nils Dijk 0c60449d08
expand shard model for geo distribution 2021-11-09 14:30:24 +01:00
Nils Dijk a7f30003a1
update pg and citus 2021-11-09 14:29:39 +01:00
Nils Dijk 695cc48cc2
add broadcast optimization rule 2021-11-09 14:29:39 +01:00
Nils Dijk f1d26e5bda
easily add new join optimization functions 2021-11-09 14:29:39 +01:00
Nils Dijk 158bd86e71
fix subtle bug in adding to a list we are iterating over 2021-11-09 14:29:39 +01:00
Nils Dijk 547f0a57c9
add paths for joins instead of rewriting them 2021-11-09 14:29:38 +01:00
Nils Dijk d5e8bda888
fix refactor and renames 2021-11-09 14:29:38 +01:00
Nils Dijk 1f97b5443a
fix targetlist to prevent projection that failed 2021-11-09 14:29:38 +01:00
Nils Dijk 57842ef81e
simple group by push down 2021-11-09 14:29:38 +01:00
Nils Dijk 47138eca8c
propagate partition value during joins 2021-11-09 14:29:38 +01:00
Nils Dijk 9b92976c4a
implement beginning of shard pruning 2021-11-09 14:29:37 +01:00
Nils Dijk debe639b4d
Add support for optimising nested loop a bit (primary key lookup). 2021-11-09 14:29:37 +01:00
Nils Dijk 38c1f0d32f
Fix misread of colocation id 2021-11-09 14:29:37 +01:00
Nils Dijk ab701669f4
Implement beginnings of join pushdown 2021-11-09 14:29:37 +01:00
Nils Dijk 0563201319
Record the colocation id on the DU 2021-11-09 14:29:37 +01:00
Nils Dijk 37721ef0cd
Define a custom path structure 2021-11-09 14:29:37 +01:00
Nils Dijk cb5d6e038c
Remove RTE reference from DU 2021-11-09 14:29:36 +01:00
Nils Dijk 174713fba9
Use path object to recreate range table entry etc 2021-11-09 14:29:36 +01:00
Nils Dijk a4ec8516fa
move path based planner to its own files 2021-11-09 14:29:36 +01:00
Nils Dijk a8097ab28d
hacky hack for double distributed join 2021-11-09 14:29:36 +01:00
Nils Dijk ff81b0b77f
support multiple remote scans 2021-11-09 14:29:36 +01:00
Nils Dijk d7e50170a9
recreate query struct for sub paths 2021-11-09 14:29:35 +01:00
Nils Dijk 7304f2c21e
beginning of set_rel_pathlist_hook to plan query on distributed table 2021-11-09 14:29:34 +01:00
Önder Kalacı 6f5a343ff4
Make sure that enterprise tests pass (#5451) 2021-11-08 18:11:19 +03:00
Önder Kalacı 98ca6ba6ca
Allow lock_shard_resources to be called by the users with privileges (#5441) 
Before this commit, we required the user to be owner of the shard/table
in order to call lock_shard_resources.

However, that is too restrictive. We can have users with GRANTS
to the table who are not owners of the tables/shards.

With this commit, we allow such patterns.
 2021-11-08 15:36:51 +01:00
Hanefi Onaldi db613b2f5c
Merge pull request #5448 from citusdata/changelog-9.5.10 2021-11-08 16:50:12 +03:00
Hanefi Onaldi 7b63edfc83
Add changelog entries for 9.5.10 2021-11-08 16:41:47 +03:00
Önder Kalacı 3bce4d76d3
Merge pull request #5405 from citusdata/simplify_executor_locks 
Simplify/Unify executor locks
 2021-11-08 13:58:11 +01:00
Onder Kalaci d5e89b1132 Unify distributed execution logic for single replicated tables 
Citus does not acquire any executor locks for shard replication == 1.
With this commit, we unify this decision and exit early.
 2021-11-08 13:52:20 +01:00
Hanefi Onaldi 20f3248b6e
Merge pull request #5445 from citusdata/changelog-9.5.9 2021-11-08 14:09:53 +03:00
Hanefi Onaldi 3d49cbf9ab
Add changelog entries for 9.5.9 2021-11-08 13:19:10 +03:00
Önder Kalacı 65911ce162
Merge pull request #5397 from citusdata/naisila/fix-partitioned-index 
Run fix_partition_shard_index_names after each wrong naming command
 2021-11-08 11:09:08 +01:00
Önder Kalacı d5b371b2e0
Merge branch 'master' into naisila/fix-partitioned-index 2021-11-08 10:53:16 +01:00
Marco Slot 7f162ba834
Merge pull request #5444 from citusdata/marcocitus/remove-master_append_table_to_shard 2021-11-08 10:49:17 +01:00
naisila 385ba94d15 Run fix_partition_shard_index_names after each wrong naming command 2021-11-08 10:43:34 +01:00
Marco Slot 78866df13c Remove master_append_table_to_shard UDF 2021-11-08 10:43:24 +01:00
Marco Slot ee0cd75648
Merge pull request #5399 from citusdata/marcocitus/remove-append-copy 2021-11-07 21:09:26 +01:00
Marco Slot fba93df4b0 Remove copy into new append shard logic 2021-11-07 21:01:40 +01:00
Marco Slot 27ba19f7e1 Fix a flappy test in drop_column_partitioned_table 2021-11-07 18:25:44 +01:00
Nils Dijk 3fcb456381
Refactor/partitioned result destreceiver (#5432) 
This change creates a slightly higher abstraction of the `PartitionedResultDestReceiver` where it decouples the partitioning from writing it to a file. This allows for easier reuse for other `DestReceiver`'s that would like to route different tuples to different `DestReceiver`'s.

Originally there was a lot of state kept in `PartitionedResultDestReceiver` to be able to lazily create `FileDestReceivers` when the first tuple arrived for that target. This convoluted the implementation of the processing of tuples with where they should go.

This refactor changes that where it makes the `PartitionedResultDestReceiver` completely agnostic of what kind of Receivers it is writing to. When constructed you pass it a list of `DestReceiver` compatible pointers with the length of `partitionCount`. Internally the `PartitionedResultDestReceiver` keeps track of which `DestReceiver`'s have been started or not, and start them when they first receive a tuple.

Alternatively, if the instantiating code of the `PartitionedResultDestReceiver` wants, the startup can be turned from lazily to eagerly. When the startup is eager (not lazy) all `rStartup` functions on the list of `DestReceiver`'s are called during the startup of the `PartitionedResultDestReceiver` and marked as such.

A downside of this approach is the following. On highly partitioned destinations we now need to allocate a `FileDestReceiver` for every target, _always_. When the data passed into the `PartitionedResultDestReceiver` is highly skewed to a small set of `FileDestReceiver`'s this will waste some memory. Given the small size of a `FileDestReceiver`, and the fact that actual file handles are only created during the processing of the startup of the `FileDestReceiver` I think this memory waste is not a problem. If this would become a problem we could refactor the source list into some kind of generator object which can generate the `DestReceiver`'s on the fly.
 2021-11-05 13:31:18 +01:00
Nils Dijk 0e7cf9f0ca
reinstate optimization that got unintentionally broken in 366461ccdb (#5418) 
DESCRIPTION: Reinstate optimisation for uniform shard interval ranges

During a refactor introduced in #4132 the following change was made, which made the optimisation in `CalculateUniformHashRangeIndex` unreachable: 
366461ccdb (diff-565a339ed3c78bc5a0d4ffeb4e91032150b1dffbeeff59cd3e65981d20b998c7L319-R319)

This PR reinstates the path to the optimisation!
 2021-11-05 13:07:51 +01:00
Önder Kalacı 763176a4d9
Some minor improvements on top of 5314 (#5428) 
* Refactor some checks in citus local tables

* all existing citus local tables are auto converted after upgrade

* Update warning messages in CreateCitusLocalTable

* Hide notice msg for auto converting local tables

* Hide hint msg

Co-authored-by: Ahmet Gedemenli <afgedemenli@gmail.com>
 2021-11-05 13:59:13 +03:00
Sait Talha Nisanci ab29c25658 Fix missing from entry 2021-11-04 18:54:52 +03:00
Halil Ozan Akgül a23f1fb259
Merge pull request #5417 from citusdata/fix_isolation_schedule_with_mx 
Turns mx on in isolations tests
 2021-11-04 17:18:50 +03:00
Halil Ozan Akgul a8f3f712cc Turns mx on in isolations tests 2021-11-04 17:12:30 +03:00
Ahmet Gedemenli b30ed46068
Fixes ALTER STATISTICS IF EXISTS bug (#5435) 
* Fix ALTER STATISTICS IF EXISTS bug
 2021-11-04 16:14:05 +03:00
Onur Tirtir 7597e5aee9
Merge pull request #5433 from citusdata/cl-928 
Add changelog for 9.2.8
 2021-11-04 15:08:01 +03:00
Onur Tirtir 4b598da672 Add changelog for 9.2.8 2021-11-04 14:56:29 +03:00