This change was actually already intended in #3124. However, the
postgres Makefile manually enables this warning too. This way we undo
that.
To confirm that it works two functions were changed to make use of not
having the warning anymore.
Phase 1 seeks to implement minimal infrastructure, so does not include:
- dynamic generation of support aggregates to handle multiple arguments
- configuration methods to direct aggregation strategy,
or mark an aggregate's serialize/deserialize as safe to operate across nodes
Aggregates can be distributed when:
- they have a single argument
- they have a combinefunc
- their transition type is not a pseudotype
This is necassery to support Q20 of the CHbenCHmark: #2582.
To summarize the fix: The subquery is converted into an INNER JOIN on a
table. This fixes the issue, since an INNER JOIN on a table is already
supported by the repartion planner.
The way this replacement is happening.:
1. Postgres replaces `col in (subquery)` with a SEMI JOIN (subquery) on col = subquery_result
2. If this subquery is simple enough Postgres will replace it with a
regular read from a table
3. If the subquery returns unique results (e.g. a primary key) Postgres
will convert the SEMI JOIN into an INNER JOIN during the planning. It
will not change this in the rewritten query though.
4. We check if Postgres sends us any SEMI JOINs during its join order
planning, if it doesn't we replace all SEMI JOINs in the rewritten
query with INNER JOIN (which we already support).
Since we've removed the executor, we don't need the specific tests.
Since the tests are already using adaptive executor, they were passing.
But, we've plenty of extra tests for adaptive executor, so seems safe
to remove.
Postgres doesn't require you to add all columns that are in the target list to
the GROUP BY when you group by a unique column (or columns). It even actively
removes these group by clauses when you do.
This is normally fine, but for repartition joins it is not. The reason for this
is that the temporary tables don't have these primary key columns. So when the
worker executes the query it will complain that it is missing columns in the
group by.
This PR fixes that by adding an ANY_VALUE aggregate around each variable in
the target list that does is not contained in the group by or in an aggregate.
This is done only for repartition joins.
The ANY_VALUE aggregate chooses the value from an undefined row in the
group.
It looks like the logic to prevent RETURNING in reference tables to
have duplicate entries that comes from local and remote executions
leads to missing some tuples for distributed tables.
With this PR, we're ensuring to kick in the logic for reference tables
only.
* Remove unused executor codes
All of the codes of real-time executor. Some functions
in router executor still remains there because there
are common functions. We'll move them to accurate places
in the follow-up commits.
* Move GUCs to transaction mngnt and remove unused struct
* Update test output
* Get rid of references of real-time executor from code
* Warn if real-time executor is picked
* Remove lots of unused connection codes
* Removed unused code for connection restrictions
Real-time and router executors cannot handle re-using of the existing
connections within a transaction block.
Adaptive executor and COPY can re-use the connections. So, there is no
reason to keep the code around for applying the restrictions in the
placement connection logic.
This makes three warnings hard errors:
1. `implicit-int`
> Warn when a declaration does not specify a type.
2. `implicit-function-declaration`
> Give a warning whenever a function is used before being declared.
3. `return-type`
> Warn whenever a function is defined with a return type that defaults to
> "int". Also warn about any "return" statement with no return value in
> a function whose return type is not "void" (falling off the end of the
> function body is considered returning without a value).
>
> For C only, warn about a "return" statement with an expression in a
> function whose return type is "void", unless the expression type is
> also "void".
The compiler behaviour when these warnings occur is not the behaviour
the developer expects. So even during development it makes sense that
they are errors.
We've changed the logic for pulling RTE_RELATIONs in #3109 and
non-colocated subquery joins and partitioned tables.
@onurctirtir found this steps where I traced back and found the issues.
While looking into it in more detail, we decided to expand the list in a
way that the callers get all the relevant RTE_RELATIONs RELKIND_RELATION,
RELKIND_PARTITIONED_TABLE, RELKIND_FOREIGN_TABLE and RELKIND_MATVIEW.
These are all relation kinds that Citus planner is aware of.
When citus.enable_repartition_joins guc is set to on, and we have
adaptive executor, there was a typo in the debug message, which was
saying realtime executor no adaptive executor.
One case where this would have been useful recently was when we changed
the `citus--9.0-1--9.1-1.sql` file to `citus--9.0-2--9.1-1.sql`. If you
had the old file installed it would never be cleaned. Any changes in
the 9.0-2 version would not be applied, because postgres tries to find
the shortest migration path, so it would always run the 9.0-1--9.1-1
version, because then it could skip 9.0-1--9.0-2.
See #3125 for details on each item.
* Remove real-time/router executor tests-1
These are the ones which doesn't have '_%d' in the test
output files.
* Remove real-time/router executor tests-2
These are the ones which has in the test
output files.
* Move the tests outputs to correct place
* Make sure that single shard commits use 2PC on adaptive executor
It looks like we've messed the tests in #2891. Fixing back.
* Use adaptive executor for all router queries
This becomes important because when task-tracker is picked, we
used to pick router executor, which doesn't make sense.
* Remove explicit references to real-time/router executors in the tests
* JobExecutorType never picks real-time/router executors
* Make sure to go incremental in test output numbers
* Even users cannot pick real-time anymore
* Do not use real-time/router custom scans
* Get rid of unnecessary normalizations
* Reflect unneeded normalizations
* Get rid of unnecessary test output file
When adding a `-Wno-some-error` flag, instead of trying to check for
`-Wno-some-error` support this checks for `-Wsome-error` support.
This is done because currently there's the a problem when compiling
citus with `./configure CFLAGS=-Werror`. If there's a warning (which is
then converted to an error) in addition to this error, the output will contain
the following message with GCC 7:
```
cc1: error: unrecognized command line option ‘-Wno-gnu-variable-sized-type-not-at-end’ [-Werror]
```
This is because of the following behaviour of GCC in case of unknown warnings:
> When an unrecognized warning option is requested (e.g., -Wunknown-warning), GCC
> emits a diagnostic stating that the option is not recognized. However, if the
> -Wno- form is used, the behavior is slightly different: no diagnostic is
> produced for -Wno-unknown-warning unless other diagnostics are being produced.
> This allows the use of new -Wno- options with old compilers, but if something
> goes wrong, the compiler warns that an unrecognized option is present.
By changing the check to `-Wsome-error`, the check will actually fail
when the warning is not supported. Instead of silently being ignored
when checking, but then coming up when another error happens.
Everything supports this, works in clang 6 and gcc 4.7
Also removes -Wdeclaration-after-statement, since declarations
after statements are only not supported in ISO C90.
Fixes#3122
This completely hides `ListCell` to the user of the loop
Example usage:
```c
WorkerNode *workerNode = NULL;
foreach_ptr(workerNode, workerNodeList) {
// Do stuff with workerNode
}
```
Instead of:
```c
ListCell *workerNodeCell = NULL;
foreach(cell, workerNodeList) {
WorkerNode *workerNode = lfirst(workerNodeCell);
// Do stuff with workerNode
}
```
Hadi Moshayedi
SaitTalhaNisanci
Jelte Fennema
Philip Dubé
Önder Kalacı
Hadi Moshayedi
Onur TIRTIR
Marco Slot
Marco Slot