external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,446 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

6446 Commits (5cedee7a7df89c862c3e36938af6bf7612c1e0a0)

Author SHA1 Message Date
Gokhan Gulbiz 5cedee7a7d
Add comment chars escaping 2023-03-15 08:09:24 +03:00
Gokhan Gulbiz 3a435e7c14
Fix attribute prefix 2023-03-15 08:08:33 +03:00
Gokhan Gulbiz c241af1eb9
Fix tenant statistics annotations normalization 2023-03-13 14:32:16 +03:00
Gokhan Gulbiz 596954622c
Indent 2023-03-13 14:31:41 +03:00
Gokhan Gulbiz 59990addd1
Validate attribute prefix existance on query string 2023-03-13 13:24:27 +03:00
Gokhan Gulbiz 0de2ebad3f
Validate input string length 2023-03-13 12:44:57 +03:00
Gokhan Gulbiz c7af2ff0ef
Normalize multiline sql comment statements 2023-03-13 12:36:02 +03:00
Gokhan Gulbiz d85c277404
Use strncpy_s instead of strncpy 2023-03-13 11:36:54 +03:00
Gokhan Gulbiz 81cf3cf408
Use palloc instead of malloc 2023-03-13 10:13:28 +03:00
Gokhan Gulbiz 7f0c502660
Replace strcpy with strcpy_s 2023-03-10 16:35:21 +03:00
Gokhan Gulbiz 64b6d7a44e
Merge remote-tracking branch 'upstream/multi-tenant-monitoring' into multi-tenant-monitoring-annotation-parsing 2023-03-10 16:29:12 +03:00
Halil Ozan Akgul fa181a8bfd fix indent 2023-03-10 12:38:56 +03:00
Halil Ozan Akgul e8d516f0f0 fix strcpy_s 2023-03-10 12:32:50 +03:00
Halil Ozan Akgul 4289d4a981 safe functions 2023-03-10 12:20:15 +03:00
Halil Ozan Akgul e32a24e94a fix 2023-03-10 11:20:44 +03:00
Halil Ozan Akgul 1249606972 fix indent 2023-03-10 11:15:27 +03:00
Halil Ozan Akgul ba32f827a9 fix 2023-03-10 11:14:05 +03:00
Halil Ozan Akgul c1a67c9aca Fix lock 2023-03-10 11:00:09 +03:00
Halil Ozan Akgul 2b55ecc167 declaration fix 2023-03-09 12:22:04 +03:00
Halil Ozan Akgul 6ea9d00971 new line 2023-03-09 12:19:29 +03:00
Halil Ozan Akgul e9017b085b indent 2023-03-09 12:16:00 +03:00
Halil Ozan Akgul e8b31058e7 open locks 2023-03-09 12:12:28 +03:00
Halil Ozan Akgul 963712897b failure 2 2023-03-09 11:54:57 +03:00
Halil Ozan Akgul ea9cb77569 failure 1 2023-03-09 11:39:03 +03:00
Gokhan Gulbiz 6b46403675
Introduce JSON based annotation parsing 2023-03-09 11:20:55 +03:00
Gokhan Gulbiz d7dee1d001
Add ExtractFieldInt32(..) to jsonbutils 2023-03-09 11:20:32 +03:00
Halil Ozan Akgul 08d82ce8bb upgrade list 2023-03-09 11:18:46 +03:00
Halil Ozan Akgul 2bf52adb35 downgrade queries 2023-03-09 11:02:28 +03:00
Halil Ozan Akgul b41e848aa6 dnm3 2023-03-08 17:13:46 +03:00
Halil Ozan Akgul a5d0e4561c dnm2 2023-03-08 16:50:30 +03:00
Halil Ozan Akgul cb39c1f677 dnm 2023-03-08 16:32:50 +03:00
Halil Ozan Akgul ff66234ca2 try without locks 2023-03-07 15:33:52 +03:00
Halil Ozan Akgul 4816817c9b fix 2023-03-06 17:30:40 +03:00
Halil Ozan Akgul 01eb68a7c9 indent 2023-03-06 17:25:30 +03:00
Halil Ozan Akgul 4ebac440f9 Minor fix 2023-03-06 17:25:30 +03:00
Halil Ozan Akgul f7a479d2e4 Add locks and convert to static shared memory 2023-03-06 17:25:30 +03:00
Halil Ozan Akgul 4dfb9c6694 Add multi tenant monitoring 2023-03-06 17:25:30 +03:00
Nils Dijk 552b7f59b8 prototype tenant attributable cpu util 2023-03-06 17:25:30 +03:00
Ahmet Gedemenli 03f1bb70b7
Rebalance shard groups with placement count less than worker count (#6739) 
DESCRIPTION: Adds logic to distribute unbalanced shards

If the number of shard placements (for a colocation group) is less than
the number of workers, it means that some of the workers will remain
empty. With this PR, we consider these shard groups as a colocation
group, in order to make them be distributed evenly as much as possible
across the cluster.

Example:
```sql
create table t1 (a int primary key);
create table t2 (a int primary key);
create table t3 (a int primary key);
set citus.shard_count =1;
select create_distributed_table('t1','a');
select create_distributed_table('t2','a',colocate_with=>'t1');
select create_distributed_table('t3','a',colocate_with=>'t2');

create table tb1 (a bigint);
create table tb2 (a bigint);
select create_distributed_table('tb1','a');
select create_distributed_table('tb2','a',colocate_with=>'tb1');

select citus_add_node('localhost',9702);
select rebalance_table_shards();
```

Here we have two colocation groups, each with one shard group. Both
shard groups are placed on the first worker node. When we add a new
worker node and try to rebalance table shards, the rebalance planner
considers it well balanced and does nothing. With this PR, the
rebalancer tries to distribute these shard groups evenly across the
cluster as much as possible. For this example, with this PR, the
rebalancer moves one of the shard groups to the second worker node.

fixes: #6715
 2023-03-06 14:14:27 +03:00
Emel Şimşek ed7cc8f460
Remove unused lock functions (#6747) 
Code cleanup. This change removes two unused functions seemingly left
over after a previous refactoring of shard move code.
 2023-03-06 13:59:45 +03:00
Jelte Fennema b489d763e1
Use pg_total_relation_size in citus_shards (#6748) 
DESCRIPTION: Correctly report shard size in citus_shards view

When looking at citus_shards, people are interested in the actual size
that all the data related to the shard takes up on disk.
`pg_total_relation_size` is the function to use for that purpose. The
previously used `pg_relation_size` does not include indexes or TOAST.
Especially the missing toast can have enormous impact on the size of the
shown data.
 2023-03-06 10:53:12 +01:00
Gledis Zeneli dc7fa0d5af
Fix multiple output version arbitrary config tests (#6744) 
With this small change, arbitrary config tests can have multiple acceptable correct outputs.

For an arbitrary config tests named `t`, now you can define `expected/t.out`, `expected/t_0.out`, `expected/t_1.out` etc and the test will succeed if the output of `sql/t.sql` is equal to any of the `t.out` or `t_{0, 1, ...}.out` files.
 2023-03-03 21:06:59 +03:00
Onur Tirtir 0d401344c2
Stabilize single node tests (#6741) 
First of all, we set next_shard_id for single_node_truncate.sql
because shard ids in the test output were changing whenever we
modify a prior test file, such as single_node.sql.

Then the flaky test detector started complaining about
single_node_truncate.sql. We fix that by specifying the correct
test dependency for it in run_test.py. We also do the same for
single_node.sql.
 2023-03-03 17:17:08 +03:00
Onur Tirtir a9820e96a3 Make single_node_truncate.sql re-runnable 
First of all, this commit sets next_shard_id for
single_node_truncate.sql because shard ids in the test output were
changing whenever we modify a prior test file.

Then the flaky test detector started complaining about
single_node_truncate.sql. We fix that by specifying the correct
test dependency for it in run_test.py.
 2023-03-02 16:33:18 +03:00
Onur Tirtir 40105bf1fc Make single_node.sql re-runnable 2023-03-02 16:33:17 +03:00
Gokhan Gulbiz f027a47ca8
Fix string eval bug in migration files check (#6740) 2023-03-02 08:44:57 +03:00
aykut-bozkurt e2654deeae
fix memory leak during altering distributed table with a lot of partition and shards (#6726) 
2 improvements to prevent memory leaks during altering or undistributing
distributed tables with a lot of partitions and shards:

1. Free memory for each call to ConvertTable so that colocated and partition tables at
`AlterDistributedTable`, `UndistributeTable`, or
`AlterTableSetAccessMethod` will not cause an increase
in memory usage,
2. Free memory while executing attach partition commands for each partition table at
`AlterDistributedTable` to prevent an increase in memory usage.

DESCRIPTION: Fixes memory leak issue during altering distributed table
with a lot of partition and shards.

Fixes https://github.com/citusdata/citus/issues/6503.
 2023-02-28 21:23:41 +03:00
Jelte Fennema 17ad61678f
Make run_test.py and create_test.py importable without errors (#6736) 
Allowing scripts to be importable is good practice in general and it's
required for the pytest testing framework that I'm adding in a follow up
PR.
 2023-02-28 00:34:42 +03:00
Jelte Fennema c018e29bec
Don't blanket ignore flake8 E402 error (#6734) 
Instead this starts ignoring it in specific places only, because most
files don't actually need it ignored.
 2023-02-27 18:17:15 +03:00
Gürkan İndibay 7b8e614039
Fixes bookworm packaging pipeline problem (#6737) 
Recently, I changed Python execution structure into virtual. Therefore,
now there is no need change built in python for the images. Since Github
is provisioning images with specific permissions, this issue caused
error.
With this PR, I removed unnecessary installation of pip and setuptools
in container docker image
Additionally, removed some unnecessary sudos and used ap-get instead of
apt in one place
 2023-02-27 15:28:36 +03:00