Add changelog entries for 11.1.2 (#6386 )

(cherry picked from commit 17cf137c4c)
Bump citus version to 11.1.2
2022-09-30 13:07:28 +03:00 · 2022-09-30 13:06:12 +03:00 · 2022-09-26 16:53:38 +02:00 · 2022-09-26 16:53:30 +02:00 · 2022-09-23 13:55:51 +03:00 · 2022-09-23 13:53:04 +03:00
1633 changed files with 55674 additions and 176350 deletions
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
--- a/.codeclimate.yml
+++ b/.codeclimate.yml
@ -1,7 +0,0 @@
 exclude_patterns:
  - "src/backend/distributed/utils/citus_outfuncs.c"
  - "src/backend/distributed/deparser/ruleutils_*.c"
  - "src/include/distributed/citus_nodes.h"
  - "src/backend/distributed/safeclib"
  - "src/backend/columnar/safeclib"
  - "**/vendor/"
--- a/.devcontainer/.gdbinit
+++ b/.devcontainer/.gdbinit
@ -1,33 +0,0 @@
 # gdbpg.py contains scripts to nicely print the postgres datastructures
 # while in a gdb session. Since the vscode debugger is based on gdb this
 # actually also works when debugging with vscode. Providing nice tools
 # to understand the internal datastructures we are working with.
 source /root/gdbpg.py
 # when debugging postgres it is convenient to _always_ have a breakpoint
 # trigger when an error is logged. Because .gdbinit is sourced before gdb
 # is fully attached and has the sources loaded. To make sure the breakpoint
 # is added when the library is loaded we temporary set the breakpoint pending
 # to on. After we have added out breakpoint we revert back to the default
 # configuration for breakpoint pending.
 # The breakpoint is hard to read, but at entry of the function we don't have
 # the level loaded in elevel. Instead we hardcode the location where the
 # level of the current error is stored. Also gdb doesn't understand the
 # ERROR symbol so we hardcode this to the value of ERROR. It is very unlikely
 # this value will ever change in postgres, but if it does we might need to
 # find a way to conditionally load the correct breakpoint.
 set breakpoint pending on
 break elog.c:errfinish if errordata[errordata_stack_depth].elevel == 21
 set breakpoint pending auto
 echo \n
 echo ----------------------------------------------------------------------------------\n
 echo when attaching to a postgres backend a breakpoint will be set on elog.c:errfinish \n
 echo it will only break on errors being raised in postgres \n
 echo \n
 echo to disable this breakpoint from vscode run `-exec disable 1` in the debug console \n
 echo this assumes it's the first breakpoint loaded as it is loaded from .gdbinit \n
 echo this can be verified with `-exec info break`, enabling can be done with \n
 echo `-exec enable 1` \n
 echo ----------------------------------------------------------------------------------\n
 echo \n
--- a/.devcontainer/.gitignore
+++ b/.devcontainer/.gitignore
@ -1 +0,0 @@
 postgresql-*.tar.bz2
--- a/.devcontainer/.psqlrc
+++ b/.devcontainer/.psqlrc
@ -1,7 +0,0 @@
 \timing on
 \pset linestyle unicode
 \pset border 2
 \setenv PAGER 'pspg --no-mouse -bX --no-commandbar --no-topbar'
 \set HISTSIZE 100000
 \set PROMPT1 '\n%[%033[1m%]%M %n@%/:%> (PID: %p)%R%[%033[0m%]%# '
 \set PROMPT2 '  '
--- a/.devcontainer/.vscode/Pipfile
+++ b/.devcontainer/.vscode/Pipfile
@ -1,12 +0,0 @@
 [[source]]
 url = "https://pypi.org/simple"
 verify_ssl = true
 name = "pypi"
 [packages]
 docopt = "*"
 [dev-packages]
 [requires]
 python_version = "3.9"
--- a/.devcontainer/.vscode/Pipfile.lock
+++ b/.devcontainer/.vscode/Pipfile.lock
@ -1,28 +0,0 @@
 {
    "_meta": {
        "hash": {
            "sha256": "6956a6700ead5804aa56bd597c93bb4a13f208d2d49d3b5399365fd240ca0797"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.9"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "docopt": {
            "hashes": [
                "sha256:49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491"
            ],
            "index": "pypi",
            "version": "==0.6.2"
        }
    },
    "develop": {}
 }
--- a/.devcontainer/.vscode/generate_c_cpp_properties-json.py
+++ b/.devcontainer/.vscode/generate_c_cpp_properties-json.py
@ -1,84 +0,0 @@
 #! /usr/bin/env pipenv-shebang
 """Generate C/C++ properties file for VSCode.
 Uses pgenv to iterate postgres versions and generate
 a C/C++ properties file for VSCode containing the
 include paths for the postgres headers.
 Usage:
  generate_c_cpp_properties-json.py <target_path>
  generate_c_cpp_properties-json.py (-h | --help)
  generate_c_cpp_properties-json.py --version
 Options:
  -h --help     Show this screen.
  --version     Show version.
 """
 import json
 import subprocess
 from docopt import docopt
 def main(args):
    target_path = args['<target_path>']
    output = subprocess.check_output(['pgenv', 'versions'])
    # typical output is:
    #      14.8      pgsql-14.8
    #  *   15.3      pgsql-15.3
    #      16beta2    pgsql-16beta2
    # where the line marked with a * is the currently active version
    #
    # we are only interested in the first word of each line, which is the version number
    # thus we strip the whitespace and the * from the line and split it into words
    # and take the first word
    versions = [line.strip('* ').split()[0] for line in output.decode('utf-8').splitlines()]
    # create the list of configurations per version
    configurations = []
    for version in versions:
        configurations.append(generate_configuration(version))
    # create the json file
    c_cpp_properties = {
        "configurations": configurations,
        "version": 4
    }
    # write the c_cpp_properties.json file
    with open(target_path, 'w') as f:
        json.dump(c_cpp_properties, f, indent=4)
 def generate_configuration(version):
    """Returns a configuration for the given postgres version.
    >>> generate_configuration('14.8')
    {
        "name": "Citus Development Configuration - Postgres 14.8",
        "includePath": [
            "/usr/local/include",
            "/home/citus/.pgenv/src/postgresql-14.8/src/**",
            "${workspaceFolder}/**",
            "${workspaceFolder}/src/include/",
        ],
        "configurationProvider": "ms-vscode.makefile-tools"
    }
    """
    return {
        "name": f"Citus Development Configuration - Postgres {version}",
        "includePath": [
            "/usr/local/include",
            f"/home/citus/.pgenv/src/postgresql-{version}/src/**",
            "${workspaceFolder}/**",
            "${workspaceFolder}/src/include/",
        ],
        "configurationProvider": "ms-vscode.makefile-tools"
    }
 if __name__ == '__main__':
    arguments = docopt(__doc__, version='0.1.0')
    main(arguments)
--- a/.devcontainer/.vscode/launch.json
+++ b/.devcontainer/.vscode/launch.json
@ -1,40 +0,0 @@
 {
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Attach Citus (devcontainer)",
            "type": "cppdbg",
            "request": "attach",
            "processId": "${command:pickProcess}",
            "program": "/home/citus/.pgenv/pgsql/bin/postgres",
            "additionalSOLibSearchPath": "/home/citus/.pgenv/pgsql/lib",
            "setupCommands": [
                {
                    "text": "handle SIGUSR1 noprint nostop pass",
                    "description": "let gdb not stop when SIGUSR1 is sent to process",
                    "ignoreFailures": true
                }
            ],
        },
        {
            "name": "Open core file",
            "type": "cppdbg",
            "request": "launch",
            "program": "/home/citus/.pgenv/pgsql/bin/postgres",
            "coreDumpPath": "${input:corefile}",
            "cwd": "${workspaceFolder}",
            "MIMode": "gdb",
        }
    ],
    "inputs": [
        {
            "id": "corefile",
            "type": "command",
            "command": "extension.commandvariable.file.pickFile",
            "args": {
                "dialogTitle": "Select core file",
                "include": "**/core*",
            },
        },
    ],
 }
--- a/.devcontainer/Dockerfile
+++ b/.devcontainer/Dockerfile
@ -1,222 +0,0 @@
 FROM ubuntu:22.04 AS base
 # environment is to make python pass an interactive shell, probably not the best timezone given a wide variety of colleagues
 ENV TZ=UTC
 RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 # install build tools
 RUN apt update && apt install -y \
    bison \
    bzip2 \
    cpanminus \
    curl \
    docbook-xml \
    docbook-xsl \
    flex \
    gcc \
    git \
    libcurl4-gnutls-dev \
    libicu-dev \
    libkrb5-dev \
    liblz4-dev \
    libpam0g-dev \
    libreadline-dev \
    libselinux1-dev \
    libssl-dev \
    libxml2-utils \
    libxslt-dev \
    libzstd-dev \
    locales \
    make \
    perl \
    pkg-config \
    python3 \
    python3-pip \
    software-properties-common \
    sudo \
    uuid-dev \
    valgrind \
    xsltproc \
    zlib1g-dev \
 && add-apt-repository ppa:deadsnakes/ppa -y \
 && apt install -y \
    python3.9-full \
 # software properties pulls in pkexec, which makes the debugger unusable in vscode
 && apt purge -y \
    software-properties-common \
 && apt autoremove -y \
 && apt clean
 RUN sudo pip3 install pipenv pipenv-shebang
 RUN cpanm install IPC::Run
 RUN locale-gen en_US.UTF-8
 # add the citus user to sudoers and allow all sudoers to login without a password prompt
 RUN useradd -ms /bin/bash citus \
 && usermod -aG sudo citus \
 && echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
 WORKDIR /home/citus
 USER citus
 # run all make commands with the number of cores available
 RUN echo "export MAKEFLAGS=\"-j \$(nproc)\"" >> "/home/citus/.bashrc"
 RUN git clone --branch v1.3.2 --depth 1 https://github.com/theory/pgenv.git .pgenv
 COPY --chown=citus:citus pgenv/config/ .pgenv/config/
 ENV PATH="/home/citus/.pgenv/bin:${PATH}"
 ENV PATH="/home/citus/.pgenv/pgsql/bin:${PATH}"
 USER citus
 # build postgres versions separately for effective parrallelism and caching of already built versions when changing only certain versions
 FROM base AS pg15
 RUN MAKEFLAGS="-j $(nproc)" pgenv build 15.13
 RUN rm .pgenv/src/*.tar*
 RUN make -C .pgenv/src/postgresql-*/ clean
 RUN make -C .pgenv/src/postgresql-*/src/include install
 # create a staging directory with all files we want to copy from our pgenv build
 # we will copy the contents of the staged folder into the final image at once
 RUN mkdir .pgenv-staging/
 RUN cp -r .pgenv/src .pgenv/pgsql-* .pgenv/config .pgenv-staging/
 RUN rm .pgenv-staging/config/default.conf
 FROM base AS pg16
 RUN MAKEFLAGS="-j $(nproc)" pgenv build 16.9
 RUN rm .pgenv/src/*.tar*
 RUN make -C .pgenv/src/postgresql-*/ clean
 RUN make -C .pgenv/src/postgresql-*/src/include install
 # create a staging directory with all files we want to copy from our pgenv build
 # we will copy the contents of the staged folder into the final image at once
 RUN mkdir .pgenv-staging/
 RUN cp -r .pgenv/src .pgenv/pgsql-* .pgenv/config .pgenv-staging/
 RUN rm .pgenv-staging/config/default.conf
 FROM base AS pg17
 RUN MAKEFLAGS="-j $(nproc)" pgenv build 17.5
 RUN rm .pgenv/src/*.tar*
 RUN make -C .pgenv/src/postgresql-*/ clean
 RUN make -C .pgenv/src/postgresql-*/src/include install
 # create a staging directory with all files we want to copy from our pgenv build
 # we will copy the contents of the staged folder into the final image at once
 RUN mkdir .pgenv-staging/
 RUN cp -r .pgenv/src .pgenv/pgsql-* .pgenv/config .pgenv-staging/
 RUN rm .pgenv-staging/config/default.conf
 FROM base AS uncrustify-builder
 RUN sudo apt update && sudo apt install -y cmake tree
 WORKDIR /uncrustify
 RUN curl -L https://github.com/uncrustify/uncrustify/archive/uncrustify-0.68.1.tar.gz | tar xz
 WORKDIR /uncrustify/uncrustify-uncrustify-0.68.1/
 RUN mkdir build
 WORKDIR /uncrustify/uncrustify-uncrustify-0.68.1/build/
 RUN cmake ..
 RUN MAKEFLAGS="-j $(nproc)" make -s
 RUN make install DESTDIR=/uncrustify
 # builder for all pipenv's to get them contained in a single layer
 FROM base AS pipenv
 WORKDIR /workspaces/citus/
 # tools to sync pgenv with vscode
 COPY --chown=citus:citus .vscode/Pipfile .vscode/Pipfile.lock .devcontainer/.vscode/
 RUN ( cd .devcontainer/.vscode && pipenv install )
 # environment to run our failure tests
 COPY --chown=citus:citus src/ src/
 RUN ( cd src/test/regress && pipenv install )
 # assemble the final container by copying over the artifacts from separately build containers
 FROM base AS devcontainer
 LABEL org.opencontainers.image.source=https://github.com/citusdata/citus
 LABEL org.opencontainers.image.description="Development container for the Citus project"
 LABEL org.opencontainers.image.licenses=AGPL-3.0-only
 RUN yes | sudo unminimize
 # install developer productivity tools
 RUN sudo apt update \
 && sudo apt install -y \
    autoconf2.69 \
    bash-completion \
    fswatch \
    gdb \
    htop \
    libdbd-pg-perl \
    libdbi-perl \
    lsof \
    man \
    net-tools \
    psmisc \
    pspg \
    tree \
    vim \
 && sudo apt clean
 # Since gdb will run in the context of the root user when debugging citus we will need to both
 # download the gdbpg.py script as the root user, into their home directory, as well as add .gdbinit
 # as a file owned by root
 # This will make that as soon as the debugger attaches to a postgres backend (or frankly any other process)
 # the gdbpg.py script will be sourced and the developer can direcly use it.
 RUN sudo curl -o /root/gdbpg.py https://raw.githubusercontent.com/tvesely/gdbpg/6065eee7872457785f830925eac665aa535caf62/gdbpg.py
 COPY --chown=root:root .gdbinit /root/
 # install developer dependencies in the global environment
 RUN --mount=type=bind,source=requirements.txt,target=requirements.txt pip install -r requirements.txt
 # for persistent bash history across devcontainers we need to have
 # a) a directory to store the history in
 # b) a prompt command to append the history to the file
 # c) specify the history file to store the history in
 # b and c are done in the .bashrc to make it persistent across shells only
 RUN sudo install -d -o citus -g citus /commandhistory \
 && echo "export PROMPT_COMMAND='history -a' && export HISTFILE=/commandhistory/.bash_history" >> "/home/citus/.bashrc"
 # install citus-dev
 RUN git clone --branch develop https://github.com/citusdata/tools.git citus-tools \
 && ( cd citus-tools/citus_dev && pipenv install ) \
 && mkdir -p ~/.local/bin \
 && ln -s /home/citus/citus-tools/citus_dev/citus_dev-pipenv .local/bin/citus_dev \
 && sudo make -C citus-tools/uncrustify install bindir=/usr/local/bin pkgsysconfdir=/usr/local/etc/ \
 && mkdir -p ~/.local/share/bash-completion/completions/ \
 && ln -s ~/citus-tools/citus_dev/bash_completion ~/.local/share/bash-completion/completions/citus_dev
 # TODO some LC_ALL errors, possibly solved by locale-gen
 RUN git clone https://github.com/so-fancy/diff-so-fancy.git \
 && mkdir -p ~/.local/bin \
 && ln -s /home/citus/diff-so-fancy/diff-so-fancy .local/bin/
 COPY --link --from=uncrustify-builder /uncrustify/usr/ /usr/
 COPY --link --from=pg15 /home/citus/.pgenv-staging/ /home/citus/.pgenv/
 COPY --link --from=pg16 /home/citus/.pgenv-staging/ /home/citus/.pgenv/
 COPY --link --from=pg17 /home/citus/.pgenv-staging/ /home/citus/.pgenv/
 COPY --link --from=pipenv /home/citus/.local/share/virtualenvs/ /home/citus/.local/share/virtualenvs/
 # place to run your cluster with citus_dev
 VOLUME /data
 RUN sudo mkdir /data \
 && sudo chown citus:citus /data
 COPY --chown=citus:citus .psqlrc .
 # with the copy linking of layers github actions seem to misbehave with the ownership of the
 # directories leading upto the link, hence a small patch layer to have to right ownerships set
 RUN sudo chown --from=root:root citus:citus -R ~
 # sets default pg version
 RUN pgenv switch 17.5
 # make connecting to the coordinator easy
 ENV PGPORT=9700
--- a/.devcontainer/Makefile
+++ b/.devcontainer/Makefile
@ -1,11 +0,0 @@
 init: ../.vscode/c_cpp_properties.json ../.vscode/launch.json
 ../.vscode:
 	mkdir -p ../.vscode
 ../.vscode/launch.json: ../.vscode .vscode/launch.json
 	cp .vscode/launch.json ../.vscode/launch.json
 ../.vscode/c_cpp_properties.json: ../.vscode
 	./.vscode/generate_c_cpp_properties-json.py ../.vscode/c_cpp_properties.json
--- a/.devcontainer/devcontainer.json
+++ b/.devcontainer/devcontainer.json
@ -1,37 +0,0 @@
 {
    "image": "ghcr.io/citusdata/citus-devcontainer:main",
    "runArgs": [
        "--cap-add=SYS_PTRACE",
        "--ulimit=core=-1",
    ],
    "forwardPorts": [
        9700
    ],
    "customizations": {
        "vscode": {
            "extensions": [
                "eamodio.gitlens",
                "GitHub.copilot-chat",
                "GitHub.copilot",
                "github.vscode-github-actions",
                "github.vscode-pull-request-github",
                "ms-vscode.cpptools-extension-pack",
                "ms-vsliveshare.vsliveshare",
                "rioj7.command-variable",
            ],
            "settings": {
                "files.exclude": {
                    "**/*.o": true,
                    "**/.deps/": true,
                }
            },
        }
    },
    "mounts": [
        "type=volume,target=/data",
        "source=citus-bashhistory,target=/commandhistory,type=volume",
    ],
    "updateContentCommand": "./configure",
    "postCreateCommand": "make -C .devcontainer/",
 }
--- a/.devcontainer/pgenv/config/default.conf
+++ b/.devcontainer/pgenv/config/default.conf
@ -1,15 +0,0 @@
 PGENV_MAKE_OPTIONS=(-s)
 PGENV_CONFIGURE_OPTIONS=(
    --enable-debug
    --enable-depend
    --enable-cassert
    --enable-tap-tests
    'CFLAGS=-ggdb -Og -g3 -fno-omit-frame-pointer -DUSE_VALGRIND'
    --with-openssl
    --with-libxml
    --with-libxslt
    --with-uuid=e2fs
    --with-icu
    --with-lz4
 )
--- a/.devcontainer/requirements.txt
+++ b/.devcontainer/requirements.txt
@ -1,9 +0,0 @@
 black==23.11.0
 click==8.1.7
 isort==5.12.0
 mypy-extensions==1.0.0
 packaging==23.2
 pathspec==0.11.2
 platformdirs==4.0.0
 tomli==2.0.1
 typing_extensions==4.8.0
--- a/.devcontainer/src/test/regress/Pipfile
+++ b/.devcontainer/src/test/regress/Pipfile
@ -1,28 +0,0 @@
 [[source]]
 name = "pypi"
 url = "https://pypi.python.org/simple"
 verify_ssl = true
 [packages]
 mitmproxy = {editable = true, ref = "main", git = "https://github.com/citusdata/mitmproxy.git"}
 construct = "*"
 docopt = "==0.6.2"
 cryptography = ">=41.0.4"
 pytest = "*"
 psycopg = "*"
 filelock = "*"
 pytest-asyncio = "*"
 pytest-timeout = "*"
 pytest-xdist = "*"
 pytest-repeat = "*"
 pyyaml = "*"
 werkzeug = "==2.3.7"
 [dev-packages]
 black = "*"
 isort = "*"
 flake8 = "*"
 flake8-bugbear = "*"
 [requires]
 python_version = "3.9"
--- a/.devcontainer/src/test/regress/Pipfile.lock
+++ b/.devcontainer/src/test/regress/Pipfile.lock
--- a/.editorconfig
+++ b/.editorconfig
@ -17,7 +17,7 @@ trim_trailing_whitespace = true
 insert_final_newline = unset
 trim_trailing_whitespace = unset
-[*.{sql,sh,py,toml}]
+[*.{sql,sh,py}]
 indent_style = space
 indent_size = 4
 tab_width = 4
--- a/.flake8
+++ b/.flake8
@ -1,7 +0,0 @@
 [flake8]
 # E203 is ignored for black
 extend-ignore = E203
 # black will truncate to 88 characters usually, but long string literals it
 # might keep. That's fine in most cases unless it gets really excessive.
 max-line-length = 150
 exclude = .git,__pycache__,vendor,tmp_*
--- a/.gitattributes
+++ b/.gitattributes
@ -25,9 +25,9 @@ configure				-whitespace
 # except these exceptions...
 src/backend/distributed/utils/citus_outfuncs.c -citus-style
 src/backend/distributed/deparser/ruleutils_13.c -citus-style
 src/backend/distributed/deparser/ruleutils_14.c -citus-style
 src/backend/distributed/deparser/ruleutils_15.c -citus-style
 src/backend/distributed/deparser/ruleutils_16.c -citus-style
 src/backend/distributed/deparser/ruleutils_17.c -citus-style
 src/backend/distributed/commands/index_pg_source.c -citus-style
 src/include/distributed/citus_nodes.h -citus-style
--- a/.github/actions/parallelization/action.yml
+++ b/.github/actions/parallelization/action.yml
@ -1,23 +0,0 @@
 name: 'Parallelization matrix'
 inputs:
  count:
    required: false
    default: 32
 outputs:
  json:
    value: ${{ steps.generate_matrix.outputs.json }}
 runs:
  using: "composite"
  steps:
    - name: Generate parallelization matrix
      id: generate_matrix
      shell: bash
      run: |-
        json_array="{\"include\": ["
        for ((i = 1; i <= ${{ inputs.count }}; i++)); do
            json_array+="{\"id\":\"$i\"},"
        done
        json_array=${json_array%,}
        json_array+=" ]}"
        echo "json=$json_array" >> "$GITHUB_OUTPUT"
        echo "json=$json_array"
--- a/.github/actions/save_logs_and_results/action.yml
+++ b/.github/actions/save_logs_and_results/action.yml
@ -1,38 +0,0 @@
 name: save_logs_and_results
 inputs:
  folder:
    required: false
    default: "log"
 runs:
  using: composite
  steps:
  - uses: actions/upload-artifact@v4.6.0
    name: Upload logs
    with:
      name: ${{ inputs.folder }}
      if-no-files-found: ignore
      path: |
        src/test/**/proxy.output
        src/test/**/results/
        src/test/**/tmp_check/master/log
        src/test/**/tmp_check/worker.57638/log
        src/test/**/tmp_check/worker.57637/log
        src/test/**/*.diffs
        src/test/**/out/ddls.sql
        src/test/**/out/queries.sql
        src/test/**/logfile_*
        /tmp/pg_upgrade_newData_logs
  - name: Publish regression.diffs
    run: |-
      diffs="$(find src/test/regress -name "*.diffs" -exec cat {} \;)"
      if ! [ -z "$diffs" ]; then
        echo '```diff' >> $GITHUB_STEP_SUMMARY
        echo -E "$diffs" >> $GITHUB_STEP_SUMMARY
        echo '```' >> $GITHUB_STEP_SUMMARY
        echo -E $diffs
      fi
    shell: bash
  - name: Print stack traces
    run: "./ci/print_stack_trace.sh"
    if: failure()
    shell: bash
--- a/.github/actions/setup_extension/action.yml
+++ b/.github/actions/setup_extension/action.yml
@ -1,35 +0,0 @@
 name: setup_extension
 inputs:
  pg_major:
    required: false
  skip_installation:
    required: false
    default: false
    type: boolean
 runs:
  using: composite
  steps:
  - name: Expose $PG_MAJOR to Github Env
    run: |-
        if [ -z "${{ inputs.pg_major }}" ]; then
          echo "PG_MAJOR=${PG_MAJOR}" >> $GITHUB_ENV
        else
          echo "PG_MAJOR=${{ inputs.pg_major }}" >> $GITHUB_ENV
        fi
    shell: bash
  - uses: actions/download-artifact@v4.1.8
    with:
      name: build-${{ env.PG_MAJOR }}
  - name: Install Extension
    if: ${{ inputs.skip_installation == 'false' }}
    run: tar xfv "install-$PG_MAJOR.tar" --directory /
    shell: bash
  - name: Configure
    run: |-
      chown -R circleci .
      git config --global --add safe.directory ${GITHUB_WORKSPACE}
      gosu circleci ./configure --without-pg-version-check
    shell: bash
  - name: Enable core dumps
    run: ulimit -c unlimited
    shell: bash
--- a/.github/actions/upload_coverage/action.yml
+++ b/.github/actions/upload_coverage/action.yml
@ -1,27 +0,0 @@
 name: coverage
 inputs:
  flags:
    required: false
  codecov_token:
    required: true
 runs:
  using: composite
  steps:
  - uses: codecov/codecov-action@v3
    with:
      flags: ${{ inputs.flags }}
      token: ${{ inputs.codecov_token }}
      verbose: true
      gcov: true
  - name: Create codeclimate coverage
    run: |-
      lcov --directory . --capture --output-file lcov.info
      lcov --remove lcov.info -o lcov.info '/usr/*'
      sed "s=^SF:$PWD/=SF:=g" -i lcov.info # relative pats are required by codeclimate
      mkdir -p /tmp/codeclimate
      cc-test-reporter format-coverage -t lcov -o /tmp/codeclimate/${{ inputs.flags }}.json lcov.info
    shell: bash
  - uses: actions/upload-artifact@v4.6.0
    with:
      path: "/tmp/codeclimate/*.json"
      name: codeclimate-${{ inputs.flags }}
--- a/.github/packaging/packaging_ignore.yml
+++ b/.github/packaging/packaging_ignore.yml
@ -1,3 +0,0 @@
 base:
  - ".* warning: ignoring old recipe for target [`']check'"
  - ".* warning: overriding recipe for target [`']check'"
--- a/.github/packaging/validate_build_output.sh
+++ b/.github/packaging/validate_build_output.sh
@ -1,51 +0,0 @@
 #!/bin/bash
 set -ex
 # Function to get the OS version
 get_rpm_os_version() {
    if [[ -f /etc/centos-release ]]; then
        cat /etc/centos-release | awk '{print $4}'
    elif [[ -f /etc/oracle-release ]]; then
        cat /etc/oracle-release | awk '{print $5}'
    else
        echo "Unknown"
    fi
 }
 package_type=${1}
 # Since $HOME is set in GH_Actions as /github/home, pyenv fails to create virtualenvs.
 # For this script, we set $HOME to /root and then set it back to /github/home.
 GITHUB_HOME="${HOME}"
 export HOME="/root"
 eval "$(pyenv init -)"
 pyenv versions
 pyenv virtualenv ${PACKAGING_PYTHON_VERSION} packaging_env
 pyenv activate packaging_env
 git clone -b v0.8.27 --depth=1  https://github.com/citusdata/tools.git tools
 python3 -m pip install -r tools/packaging_automation/requirements.txt
 echo "Package type: ${package_type}"
 echo "OS version: $(get_rpm_os_version)"
 # For RHEL 7, we need to install urllib3<2 due to below execution error
 # ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl'
 # module is compiled with 'OpenSSL 1.0.2k-fips  26 Jan 2017'.
 # See: https://github.com/urllib3/urllib3/issues/2168
 if [[ ${package_type} == "rpm" && $(get_rpm_os_version) == 7* ]]; then
    python3 -m pip uninstall -y urllib3
    python3 -m pip install 'urllib3<2'
 fi
 python3 -m tools.packaging_automation.validate_build_output --output_file output.log \
                                                            --ignore_file .github/packaging/packaging_ignore.yml \
                                                            --package_type ${package_type}
 pyenv deactivate
 # Set $HOME back to /github/home
 export HOME=${GITHUB_HOME}
 # Print the output to the console
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@ -1,546 +0,0 @@
 name: Build & Test
 run-name: Build & Test - ${{ github.event.pull_request.title || github.ref_name }}
 concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true
 on:
  workflow_dispatch:
    inputs:
      skip_test_flakyness:
        required: false
        default: false
        type: boolean
  push:
    branches:
      - "main"
      - "release-*"
  pull_request:
    types: [opened, reopened,synchronize]
  merge_group:
 jobs:
  # Since GHA does not interpolate env varibles in matrix context, we need to
  # define them in a separate job and use them in other jobs.
  params:
    runs-on: ubuntu-latest
    name: Initialize parameters
    outputs:
      build_image_name: "ghcr.io/citusdata/extbuilder"
      test_image_name: "ghcr.io/citusdata/exttester"
      citusupgrade_image_name: "ghcr.io/citusdata/citusupgradetester"
      fail_test_image_name: "ghcr.io/citusdata/failtester"
      pgupgrade_image_name: "ghcr.io/citusdata/pgupgradetester"
      style_checker_image_name: "ghcr.io/citusdata/stylechecker"
      style_checker_tools_version: "0.8.18"
      sql_snapshot_pg_version: "17.5"
      image_suffix: "-dev-d28f316"
      pg15_version: '{ "major": "15", "full": "15.13" }'
      pg16_version: '{ "major": "16", "full": "16.9" }'
      pg17_version: '{ "major": "17", "full": "17.5" }'
      upgrade_pg_versions: "15.13-16.9-17.5"
    steps:
      # Since GHA jobs need at least one step we use a noop step here.
      - name: Set up parameters
        run: echo 'noop'
  check-sql-snapshots:
    needs: params
    runs-on: ubuntu-latest
    container:
      image: ${{ needs.params.outputs.build_image_name }}:${{ needs.params.outputs.sql_snapshot_pg_version }}${{ needs.params.outputs.image_suffix }}
      options: --user root
    steps:
    - uses: actions/checkout@v4
    - name: Check Snapshots
      run: |
        git config --global --add safe.directory ${GITHUB_WORKSPACE}
        ci/check_sql_snapshots.sh
  check-style:
    needs: params
    runs-on: ubuntu-latest
    container:
      image: ${{ needs.params.outputs.style_checker_image_name }}:${{ needs.params.outputs.style_checker_tools_version }}${{ needs.params.outputs.image_suffix }}
    steps:
    - name: Check Snapshots
      run: |
        git config --global --add safe.directory ${GITHUB_WORKSPACE}
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Check C Style
      run: citus_indent --check
    - name: Check Python style
      run: black --check .
    - name: Check Python import order
      run: isort --check .
    - name: Check Python lints
      run: flake8 .
    - name: Fix whitespace
      run: ci/editorconfig.sh && git diff --exit-code
    - name: Remove useless declarations
      run: ci/remove_useless_declarations.sh && git diff --cached --exit-code
    - name: Sort and group includes
      run: ci/sort_and_group_includes.sh && git diff --exit-code
    - name: Normalize test output
      run: ci/normalize_expected.sh && git diff --exit-code
    - name: Check for C-style comments in migration files
      run: ci/disallow_c_comments_in_migrations.sh && git diff --exit-code
    - name: 'Check for comment--cached ns that start with # character in spec files'
      run: ci/disallow_hash_comments_in_spec_files.sh && git diff --exit-code
    - name: Check for gitignore entries .for source files
      run: ci/fix_gitignore.sh && git diff --exit-code
    - name: Check for lengths of changelog entries
      run: ci/disallow_long_changelog_entries.sh
    - name: Check for banned C API usage
      run: ci/banned.h.sh
    - name: Check for tests missing in schedules
      run: ci/check_all_tests_are_run.sh
    - name: Check if all CI scripts are actually run
      run: ci/check_all_ci_scripts_are_run.sh
    - name: Check if all GUCs are sorted alphabetically
      run: ci/check_gucs_are_alphabetically_sorted.sh
    - name: Check for missing downgrade scripts
      run: ci/check_migration_files.sh
  build:
    needs: params
    name: Build for PG${{ fromJson(matrix.pg_version).major }}
    strategy:
      fail-fast: false
      matrix:
        image_name:
          - ${{ needs.params.outputs.build_image_name }}
        image_suffix:
          - ${{ needs.params.outputs.image_suffix}}
        pg_version:
          - ${{ needs.params.outputs.pg15_version }}
          - ${{ needs.params.outputs.pg16_version }}
          - ${{ needs.params.outputs.pg17_version }}
    runs-on: ubuntu-latest
    container:
      image: "${{ matrix.image_name }}:${{ fromJson(matrix.pg_version).full }}${{ matrix.image_suffix }}"
      options: --user root
    steps:
    - uses: actions/checkout@v4
    - name: Expose $PG_MAJOR to Github Env
      run: echo "PG_MAJOR=${PG_MAJOR}" >> $GITHUB_ENV
      shell: bash
    - name: Build
      run: "./ci/build-citus.sh"
      shell: bash
    - uses: actions/upload-artifact@v4.6.0
      with:
        name: build-${{ env.PG_MAJOR }}
        path: |-
          ./build-${{ env.PG_MAJOR }}/*
          ./install-${{ env.PG_MAJOR }}.tar
  test-citus:
    name: PG${{ fromJson(matrix.pg_version).major }} - ${{ matrix.make }}
    strategy:
      fail-fast: false
      matrix:
        suite:
          - regress
        image_name:
          - ${{ needs.params.outputs.test_image_name }}
        pg_version:
          - ${{ needs.params.outputs.pg15_version }}
          - ${{ needs.params.outputs.pg16_version }}
          - ${{ needs.params.outputs.pg17_version }}
        make:
          - check-split
          - check-multi
          - check-multi-1
          - check-multi-mx
          - check-vanilla
          - check-isolation
          - check-operations
          - check-follower-cluster
          - check-columnar
          - check-columnar-isolation
          - check-enterprise
          - check-enterprise-isolation
          - check-enterprise-isolation-logicalrep-1
          - check-enterprise-isolation-logicalrep-2
          - check-enterprise-isolation-logicalrep-3
        include:
          - make: check-failure
            pg_version: ${{ needs.params.outputs.pg15_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-failure
            pg_version: ${{ needs.params.outputs.pg16_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-failure
            pg_version: ${{ needs.params.outputs.pg17_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-enterprise-failure
            pg_version: ${{ needs.params.outputs.pg15_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-enterprise-failure
            pg_version: ${{ needs.params.outputs.pg16_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-enterprise-failure
            pg_version: ${{ needs.params.outputs.pg17_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-pytest
            pg_version: ${{ needs.params.outputs.pg15_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-pytest
            pg_version: ${{ needs.params.outputs.pg16_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-pytest
            pg_version: ${{ needs.params.outputs.pg17_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: installcheck
            suite: cdc
            image_name: ${{ needs.params.outputs.test_image_name }}
            pg_version: ${{ needs.params.outputs.pg15_version }}
          - make: installcheck
            suite: cdc
            image_name: ${{ needs.params.outputs.test_image_name }}
            pg_version: ${{ needs.params.outputs.pg16_version }}
          - make: installcheck
            suite: cdc
            image_name: ${{ needs.params.outputs.test_image_name }}
            pg_version: ${{ needs.params.outputs.pg17_version }}
          - make: check-query-generator
            pg_version: ${{ needs.params.outputs.pg15_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-query-generator
            pg_version: ${{ needs.params.outputs.pg16_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
          - make: check-query-generator
            pg_version: ${{ needs.params.outputs.pg17_version }}
            suite: regress
            image_name: ${{ needs.params.outputs.fail_test_image_name }}
    runs-on: ubuntu-latest
    container:
      image: "${{ matrix.image_name }}:${{ fromJson(matrix.pg_version).full }}${{ needs.params.outputs.image_suffix }}"
      options: --user root --dns=8.8.8.8
      # Due to Github creates a default network for each job, we need to use
      # --dns= to have similar DNS settings as our other CI systems or local
      # machines. Otherwise, we may see different results.
    needs:
    - params
    - build
    steps:
    - uses: actions/checkout@v4
    - uses: "./.github/actions/setup_extension"
    - name: Run Test
      run: gosu circleci make -C src/test/${{ matrix.suite }} ${{ matrix.make }}
      timeout-minutes: 20
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
        folder: ${{ fromJson(matrix.pg_version).major }}_${{ matrix.make }}
    - uses: "./.github/actions/upload_coverage"
      if: always()
      with:
        flags: ${{ env.PG_MAJOR }}_${{ matrix.suite }}_${{ matrix.make }}
        codecov_token: ${{ secrets.CODECOV_TOKEN }}
  test-arbitrary-configs:
    name: PG${{ fromJson(matrix.pg_version).major }} - check-arbitrary-configs-${{ matrix.parallel }}
    runs-on: ["self-hosted", "1ES.Pool=1es-gha-citusdata-pool"]
    container:
      image: "${{ matrix.image_name }}:${{ fromJson(matrix.pg_version).full }}${{ needs.params.outputs.image_suffix }}"
      options: --user root
    needs:
      - params
      - build
    strategy:
      fail-fast: false
      matrix:
        image_name:
          - ${{ needs.params.outputs.fail_test_image_name }}
        pg_version:
          - ${{ needs.params.outputs.pg15_version }}
          - ${{ needs.params.outputs.pg16_version }}
          - ${{ needs.params.outputs.pg17_version }}
        parallel: [0,1,2,3,4,5] # workaround for running 6 parallel jobs
    steps:
    - uses: actions/checkout@v4
    - uses: "./.github/actions/setup_extension"
    - name: Test arbitrary configs
      run: |-
        # we use parallel jobs to split the tests into 6 parts and run them in parallel
        # the script below extracts the tests for the current job
        N=6  # Total number of jobs (see matrix.parallel)
        X=${{ matrix.parallel }}  # Current job number
        TESTS=$(src/test/regress/citus_tests/print_test_names.py |
          tr '\n' ',' | awk -v N="$N" -v X="$X" -F, '{
            split("", parts)
            for (i = 1; i <= NF; i++) {
                parts[i % N] = parts[i % N] $i ","
            }
            print substr(parts[X], 1, length(parts[X])-1)
        }')
        echo $TESTS
        gosu circleci \
          make -C src/test/regress \
            check-arbitrary-configs parallel=4 CONFIGS=$TESTS
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
        folder: ${{ env.PG_MAJOR }}_arbitrary_configs_${{ matrix.parallel }}
    - uses: "./.github/actions/upload_coverage"
      if: always()
      with:
        flags: ${{ env.PG_MAJOR }}_arbitrary_configs_${{ matrix.parallel }}
        codecov_token: ${{ secrets.CODECOV_TOKEN }}
  test-pg-upgrade:
    name: PG${{ matrix.old_pg_major }}-PG${{ matrix.new_pg_major }} - check-pg-upgrade
    runs-on: ubuntu-latest
    container:
      image: "${{ needs.params.outputs.pgupgrade_image_name }}:${{ needs.params.outputs.upgrade_pg_versions }}${{ needs.params.outputs.image_suffix }}"
      options: --user root
    needs:
    - params
    - build
    strategy:
      fail-fast: false
      matrix:
        include:
          - old_pg_major: 15
            new_pg_major: 16
          - old_pg_major: 16
            new_pg_major: 17
          - old_pg_major: 15
            new_pg_major: 17
    env:
      old_pg_major: ${{ matrix.old_pg_major }}
      new_pg_major: ${{ matrix.new_pg_major }}
    steps:
    - uses: actions/checkout@v4
    - uses: "./.github/actions/setup_extension"
      with:
        pg_major: "${{ env.old_pg_major }}"
    - uses: "./.github/actions/setup_extension"
      with:
        pg_major: "${{ env.new_pg_major }}"
    - name: Install and test postgres upgrade
      run: |-
        gosu circleci \
          make -C src/test/regress \
            check-pg-upgrade \
            old-bindir=/usr/lib/postgresql/${{ env.old_pg_major }}/bin \
            new-bindir=/usr/lib/postgresql/${{ env.new_pg_major }}/bin
    - name: Copy pg_upgrade logs for newData dir
      run: |-
        mkdir -p /tmp/pg_upgrade_newData_logs
        if ls src/test/regress/tmp_upgrade/newData/*.log 1> /dev/null 2>&1; then
            cp src/test/regress/tmp_upgrade/newData/*.log /tmp/pg_upgrade_newData_logs
        fi
      if: failure()
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
        folder: ${{ env.old_pg_major }}_${{ env.new_pg_major }}_upgrade
    - uses: "./.github/actions/upload_coverage"
      if: always()
      with:
        flags: ${{ env.old_pg_major }}_${{ env.new_pg_major }}_upgrade
        codecov_token: ${{ secrets.CODECOV_TOKEN }}
  test-citus-upgrade:
    name: PG${{ fromJson(needs.params.outputs.pg15_version).major }} - check-citus-upgrade
    runs-on: ubuntu-latest
    container:
      image: "${{ needs.params.outputs.citusupgrade_image_name }}:${{ fromJson(needs.params.outputs.pg15_version).full }}${{ needs.params.outputs.image_suffix }}"
      options: --user root
    needs:
    - params
    - build
    steps:
    - uses: actions/checkout@v4
    - uses: "./.github/actions/setup_extension"
      with:
        skip_installation: true
    - name: Install and test citus upgrade
      run: |-
        # run make check-citus-upgrade for all citus versions
        # the image has ${CITUS_VERSIONS} set with all verions it contains the binaries of
        for citus_version in ${CITUS_VERSIONS}; do \
          gosu circleci \
            make -C src/test/regress \
              check-citus-upgrade \
              bindir=/usr/lib/postgresql/${PG_MAJOR}/bin \
              citus-old-version=${citus_version} \
              citus-pre-tar=/install-pg${PG_MAJOR}-citus${citus_version}.tar \
              citus-post-tar=${GITHUB_WORKSPACE}/install-$PG_MAJOR.tar; \
        done;
        # run make check-citus-upgrade-mixed for all citus versions
        # the image has ${CITUS_VERSIONS} set with all verions it contains the binaries of
        for citus_version in ${CITUS_VERSIONS}; do \
          gosu circleci \
            make -C src/test/regress \
              check-citus-upgrade-mixed \
              citus-old-version=${citus_version} \
              bindir=/usr/lib/postgresql/${PG_MAJOR}/bin \
              citus-pre-tar=/install-pg${PG_MAJOR}-citus${citus_version}.tar \
              citus-post-tar=${GITHUB_WORKSPACE}/install-$PG_MAJOR.tar; \
        done;
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
        folder: ${{ env.PG_MAJOR }}_citus_upgrade
    - uses: "./.github/actions/upload_coverage"
      if: always()
      with:
        flags: ${{ env.PG_MAJOR }}_citus_upgrade
        codecov_token: ${{ secrets.CODECOV_TOKEN }}
  upload-coverage:
    # secret below is not available for forks so disabling upload action for them
    if: ${{ github.event.pull_request.head.repo.full_name == github.repository || github.event_name != 'pull_request' }}
    env:
      CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}
    runs-on: ubuntu-latest
    container:
      image: ${{ needs.params.outputs.test_image_name }}:${{ fromJson(needs.params.outputs.pg17_version).full }}${{ needs.params.outputs.image_suffix }}
    needs:
      - params
      - test-citus
      - test-arbitrary-configs
      - test-citus-upgrade
      - test-pg-upgrade
    steps:
      - uses: actions/download-artifact@v4.1.8
        with:
          pattern: codeclimate*
          path: codeclimate
          merge-multiple: true
      - name: Upload coverage results to Code Climate
        run: |-
          cc-test-reporter sum-coverage codeclimate/*.json -o total.json
          cc-test-reporter upload-coverage -i total.json
  ch_benchmark:
    name: CH Benchmark
    if: startsWith(github.ref, 'refs/heads/ch_benchmark/')
    runs-on: ubuntu-latest
    needs:
    - build
    steps:
    - uses: actions/checkout@v4
    - uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}
    - name: install dependencies and run ch_benchmark tests
      uses: azure/CLI@v1
      with:
        inlineScript: |
          cd ./src/test/hammerdb
          chmod +x run_hammerdb.sh
          run_hammerdb.sh citusbot_ch_benchmark_rg
  tpcc_benchmark:
    name: TPCC Benchmark
    if: startsWith(github.ref, 'refs/heads/tpcc_benchmark/')
    runs-on: ubuntu-latest
    needs:
    - build
    steps:
    - uses: actions/checkout@v4
    - uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}
    - name: install dependencies and run tpcc_benchmark tests
      uses: azure/CLI@v1
      with:
        inlineScript: |
          cd ./src/test/hammerdb
          chmod +x run_hammerdb.sh
          run_hammerdb.sh citusbot_tpcc_benchmark_rg
  prepare_parallelization_matrix_32:
    name: Prepare parallelization matrix
    if: ${{ needs.test-flakyness-pre.outputs.tests != ''}}
    needs: test-flakyness-pre
    runs-on: ubuntu-latest
    outputs:
      json: ${{ steps.parallelization.outputs.json }}
    steps:
      - uses: actions/checkout@v4
      - uses: "./.github/actions/parallelization"
        id: parallelization
        with:
          count: 32
  test-flakyness-pre:
    name: Detect regression tests need to be ran
    if: ${{ !inputs.skip_test_flakyness }}}
    runs-on: ubuntu-latest
    needs: build
    outputs:
      tests: ${{ steps.detect-regression-tests.outputs.tests }}
    steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0
    - name: Detect regression tests need to be ran
      id: detect-regression-tests
      run: |-
        detected_changes=$(git diff origin/main... --name-only --diff-filter=AM | (grep 'src/test/regress/sql/.*\.sql\|src/test/regress/spec/.*\.spec\|src/test/regress/citus_tests/test/test_.*\.py' || true))
        tests=${detected_changes}
        # split the tests to be skipped --today we only skip upgrade tests
        skipped_tests=""
        not_skipped_tests=""
        for test in $tests; do
            if [[ $test =~ ^src/test/regress/sql/upgrade_ ]]; then
                skipped_tests="$skipped_tests $test"
            else
                not_skipped_tests="$not_skipped_tests $test"
            fi
        done
        if [ ! -z "$skipped_tests" ]; then
            echo "Skipped tests " $skipped_tests
        fi
        if [ -z "$not_skipped_tests" ]; then
            echo "Not detected any tests that flaky test detection should run"
        else
            echo "Detected tests " $not_skipped_tests
        fi
        echo 'tests<<EOF' >> $GITHUB_OUTPUT
        echo "$not_skipped_tests" >> "$GITHUB_OUTPUT"
        echo 'EOF' >> $GITHUB_OUTPUT
  test-flakyness:
    if: ${{ needs.test-flakyness-pre.outputs.tests != ''}}
    name: Test flakyness
    runs-on: ubuntu-latest
    container:
      image: ${{ needs.params.outputs.fail_test_image_name }}:${{ fromJson(needs.params.outputs.pg17_version).full }}${{ needs.params.outputs.image_suffix }}
      options: --user root
    env:
      runs: 8
    needs:
    - params
    - build
    - test-flakyness-pre
    - prepare_parallelization_matrix_32
    strategy:
      fail-fast: false
      matrix: ${{ fromJson(needs.prepare_parallelization_matrix_32.outputs.json) }}
    steps:
    - uses: actions/checkout@v4
    - uses: actions/download-artifact@v4.1.8
    - uses: "./.github/actions/setup_extension"
    - name: Run minimal tests
      run: |-
        tests="${{ needs.test-flakyness-pre.outputs.tests }}"
        tests_array=($tests)
        for test in "${tests_array[@]}"
        do
            test_name=$(echo "$test" | sed -r "s/.+\/(.+)\..+/\1/")
            gosu circleci src/test/regress/citus_tests/run_test.py $test_name --repeat ${{ env.runs }} --use-whole-schedule-line
        done
      shell: bash
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
        folder: test_flakyness_parallel_${{ matrix.id }}
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@ -1,79 +0,0 @@
 name: "CodeQL"
 on:
  schedule:
    - cron: '59 23 * * 6'
  workflow_dispatch:
 jobs:
  analyze:
    name: Analyze
    runs-on: ubuntu-22.04
    permissions:
      actions: read
      contents: read
      security-events: write
    strategy:
      fail-fast: false
      matrix:
        language: [ 'cpp', 'python']
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
    - name: Initialize CodeQL
      uses: github/codeql-action/init@v3
      with:
        languages: ${{ matrix.language }}
    - name: Install package dependencies
      run: |
        # Create the file repository configuration:
        sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main 15" > /etc/apt/sources.list.d/pgdg.list'
        # Import the repository signing key:
        wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
        sudo apt-get update
        sudo apt-get install -y --no-install-recommends \
          autotools-dev \
          build-essential \
          ca-certificates \
          curl \
          debhelper \
          devscripts \
          fakeroot \
          flex \
          libcurl4-openssl-dev \
          libdistro-info-perl \
          libedit-dev \
          libfile-fcntllock-perl \
          libicu-dev \
          libkrb5-dev \
          liblz4-1 \
          liblz4-dev \
          libpam0g-dev \
          libreadline-dev \
          libselinux1-dev \
          libssl-dev \
          libxslt-dev \
          libzstd-dev \
          libzstd1 \
          lintian \
          postgresql-server-dev-15 \
          postgresql-server-dev-all \
          python3-pip \
          python3-setuptools \
          wget \
          zlib1g-dev
    - name: Configure, Build and Install Citus
      if: matrix.language == 'cpp'
      run: |
        ./configure
        make -sj8
        sudo make install-all
    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3
--- a/.github/workflows/devcontainer.yml
+++ b/.github/workflows/devcontainer.yml
@ -1,54 +0,0 @@
 name: "Build devcontainer"
 # Since building of containers can be quite time consuming, and take up some storage,
 # there is no need to finish a build for a tag if new changes are concurrently being made.
 # This cancels any previous builds for the same tag, and only the latest one will be kept.
 concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 on:
  push:
    paths:
      - ".devcontainer/**"
  workflow_dispatch:
 jobs:
  docker:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
      attestations: write
      id-token: write
    steps:
      -
        name: Docker meta
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: |
            ghcr.io/citusdata/citus-devcontainer
          tags: |
            type=ref,event=branch
            type=sha
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      -
        name: 'Login to GitHub Container Registry'
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{github.actor}}
          password: ${{secrets.GITHUB_TOKEN}}
      -
        name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: "{{defaultContext}}:.devcontainer"
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max
--- a/.github/workflows/flaky_test_debugging.yml
+++ b/.github/workflows/flaky_test_debugging.yml
@ -1,79 +0,0 @@
 name: Flaky test debugging
 run-name: Flaky test debugging - ${{ inputs.flaky_test }} (${{ inputs.flaky_test_runs_per_job }}x${{ inputs.flaky_test_parallel_jobs }})
 concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true
 on:
  workflow_dispatch:
    inputs:
      flaky_test:
        required: true
        type: string
        description: Test to run
      flaky_test_runs_per_job:
        required: false
        default: 8
        type: number
        description: Number of times to run the test
      flaky_test_parallel_jobs:
        required: false
        default: 32
        type: number
        description: Number of parallel jobs to run
 jobs:
  build:
    name: Build Citus
    runs-on: ubuntu-latest
    container:
      image: ${{ vars.build_image_name }}:${{ vars.pg15_version  }}${{ vars.image_suffix }}
      options: --user root
    steps:
    - uses: actions/checkout@v4
    - name: Configure, Build, and Install
      run: |
        echo "PG_MAJOR=${PG_MAJOR}" >> $GITHUB_ENV
        ./ci/build-citus.sh
      shell: bash
    - uses: actions/upload-artifact@v4.6.0
      with:
        name: build-${{ env.PG_MAJOR }}
        path: |-
          ./build-${{ env.PG_MAJOR }}/*
          ./install-${{ env.PG_MAJOR }}.tar
  prepare_parallelization_matrix:
    name: Prepare parallelization matrix
    runs-on: ubuntu-latest
    outputs:
      json: ${{ steps.parallelization.outputs.json }}
    steps:
      - uses: actions/checkout@v4
      - uses: "./.github/actions/parallelization"
        id: parallelization
        with:
          count: ${{ inputs.flaky_test_parallel_jobs }}
  test_flakyness:
    name: Test flakyness
    runs-on: ubuntu-latest
    container:
      image: ${{ vars.fail_test_image_name }}:${{ vars.pg15_version  }}${{ vars.image_suffix }}
      options: --user root
    needs:
      [build, prepare_parallelization_matrix]
    env:
      test: "${{ inputs.flaky_test }}"
      runs: "${{ inputs.flaky_test_runs_per_job }}"
      skip: false
    strategy:
      fail-fast: false
      matrix: ${{ fromJson(needs.prepare_parallelization_matrix.outputs.json) }}
    steps:
    - uses: actions/checkout@v4
    - uses: "./.github/actions/setup_extension"
    - name: Run minimal tests
      run: |-
          gosu circleci src/test/regress/citus_tests/run_test.py ${{ env.test }} --repeat ${{ env.runs }} --use-whole-schedule-line
      shell: bash
    - uses: "./.github/actions/save_logs_and_results"
      if: always()
      with:
          folder: check_flakyness_parallel_${{ matrix.id }}
--- a/.github/workflows/packaging-test-pipelines.yml
+++ b/.github/workflows/packaging-test-pipelines.yml
@ -1,177 +0,0 @@
 name: Build tests in packaging images
 on:
  pull_request:
    types: [opened, reopened,synchronize]
  merge_group:
  workflow_dispatch:
 concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 jobs:
  get_postgres_versions_from_file:
    runs-on: ubuntu-latest
    outputs:
      pg_versions: ${{ steps.get-postgres-versions.outputs.pg_versions }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 2
      - name: Get Postgres Versions
        id: get-postgres-versions
        run: |
          set -euxo pipefail
          # Postgres versions are stored in .github/workflows/build_and_test.yml
          # file in json strings with major and full keys.
          # Below command extracts the versions and get the unique values.
          pg_versions=$(cat .github/workflows/build_and_test.yml | grep -oE '"major": "[0-9]+", "full": "[0-9.]+"' | sed -E 's/"major": "([0-9]+)", "full": "([0-9.]+)"/\1/g' | sort | uniq | tr '\n', ',')
          pg_versions_array="[ ${pg_versions} ]"
          echo "Supported PG Versions: ${pg_versions_array}"
          # Below line is needed to set the output variable to be used in the next job
          echo "pg_versions=${pg_versions_array}" >> $GITHUB_OUTPUT
        shell: bash
  rpm_build_tests:
    name: rpm_build_tests
    needs: get_postgres_versions_from_file
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        # While we use separate images for different Postgres versions in rpm
        # based distros
        # For this reason, we need to use a "matrix" to generate names of
        # rpm images, e.g. citus/packaging:centos-7-pg12
        packaging_docker_image:
          - oraclelinux-8
          - almalinux-8
          - almalinux-9
        POSTGRES_VERSION: ${{ fromJson(needs.get_postgres_versions_from_file.outputs.pg_versions) }}
    container:
      image: citus/packaging:${{ matrix.packaging_docker_image }}-pg${{ matrix.POSTGRES_VERSION }}
      options: --user root
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Set Postgres and python parameters for rpm based distros
        run: |
          echo "/usr/pgsql-${{ matrix.POSTGRES_VERSION }}/bin" >> $GITHUB_PATH
          echo "/root/.pyenv/bin:$PATH" >> $GITHUB_PATH
          echo "PACKAGING_PYTHON_VERSION=3.8.16" >> $GITHUB_ENV
      - name: Configure
        run: |
          echo "Current Shell:$0"
          echo "GCC Version: $(gcc --version)"
          ./configure 2>&1 | tee output.log
      - name: Make clean
        run: |
          make clean
      - name: Make
        run: |
          git config --global --add safe.directory ${GITHUB_WORKSPACE}
          make CFLAGS="-Wno-missing-braces" -sj$(cat /proc/cpuinfo | grep "core id" | wc -l) 2>&1 | tee -a output.log
          # Check the exit code of the make command
          make_exit_code=${PIPESTATUS[0]}
          # If the make command returned a non-zero exit code, exit with the same code
          if [[ $make_exit_code -ne 0 ]]; then
              echo "make command failed with exit code $make_exit_code"
              exit $make_exit_code
          fi
      - name: Make install
        run: |
          make CFLAGS="-Wno-missing-braces" install 2>&1 | tee -a output.log
      - name: Validate output
        env:
          POSTGRES_VERSION: ${{ matrix.POSTGRES_VERSION }}
          PACKAGING_DOCKER_IMAGE: ${{ matrix.packaging_docker_image }}
        run: |
          echo "Postgres version: ${POSTGRES_VERSION}"
          ./.github/packaging/validate_build_output.sh "rpm"
  deb_build_tests:
    name: deb_build_tests
    needs: get_postgres_versions_from_file
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        # On deb based distros, we use the same docker image for
        # builds based on different Postgres versions because deb
        # based images include all postgres installations.
        # For this reason, we have multiple runs --which is 3 today--
        # for each deb based image and we use POSTGRES_VERSION to set
        # PG_CONFIG variable in each of those runs.
        packaging_docker_image:
          - debian-bookworm-all
          - debian-bullseye-all
          - ubuntu-focal-all
          - ubuntu-jammy-all
        POSTGRES_VERSION: ${{ fromJson(needs.get_postgres_versions_from_file.outputs.pg_versions) }}
    container:
      image: citus/packaging:${{ matrix.packaging_docker_image }}
      options: --user root
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Set pg_config path and python parameters for deb based distros
        run: |
          echo "PG_CONFIG=/usr/lib/postgresql/${{ matrix.POSTGRES_VERSION }}/bin/pg_config" >> $GITHUB_ENV
          echo "/root/.pyenv/bin:$PATH" >> $GITHUB_PATH
          echo "PACKAGING_PYTHON_VERSION=3.8.16" >> $GITHUB_ENV
      - name: Configure
        run: |
          echo "Current Shell:$0"
          echo "GCC Version: $(gcc --version)"
          ./configure 2>&1 | tee output.log
      - name: Make clean
        run: |
          make clean
      - name: Make
        shell: bash
        run: |
          set -e
          git config --global --add safe.directory ${GITHUB_WORKSPACE}
          make -sj$(cat /proc/cpuinfo | grep "core id" | wc -l) 2>&1 | tee -a output.log
          # Check the exit code of the make command
          make_exit_code=${PIPESTATUS[0]}
          # If the make command returned a non-zero exit code, exit with the same code
          if [[ $make_exit_code -ne 0 ]]; then
              echo "make command failed with exit code $make_exit_code"
              exit $make_exit_code
          fi
      - name: Make install
        run: |
          make install 2>&1 | tee -a output.log
      - name: Validate output
        env:
          POSTGRES_VERSION: ${{ matrix.POSTGRES_VERSION }}
          PACKAGING_DOCKER_IMAGE: ${{ matrix.packaging_docker_image }}
        run: |
          echo "Postgres version: ${POSTGRES_VERSION}"
          ./.github/packaging/validate_build_output.sh "deb"
--- a/.gitignore
+++ b/.gitignore
@ -38,9 +38,6 @@ lib*.pc
 /Makefile.global
 /src/Makefile.custom
 /compile_commands.json
 /src/backend/distributed/cdc/build-cdc-*/*
 /src/test/cdc/tmp_check/*
 # temporary files vim creates
 *.swp
@ -54,7 +51,3 @@ lib*.pc
 # style related temporary outputs
 *.uncrustify
 .venv
 # added output when modifying check_gucs_are_alphabetically_sorted.sh
 guc.out
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,909 +1,5 @@
 ### citus v13.1.0 (May 30th, 2025) ###
 * Adds `citus_stat_counters` view that can be used to query
  stat counters that Citus collects while the feature is enabled, which is
  controlled by citus.enable_stat_counters. `citus_stat_counters()` can be
  used to query the stat counters for the provided database oid and
  `citus_stat_counters_reset()` can be used to reset them for the provided
  database oid or for the current database if nothing or 0 is provided (#7917)
 * Adds `citus_nodes` view that displays the node name, port role, and "active"
  for nodes in the cluster (#7968)
 * Adds `citus_is_primary_node()` UDF to determine if the current node is a
  primary node in the cluster (#7720)
 * Adds support for propagating `GRANT/REVOKE` rights on table columns (#7918)
 * Adds support for propagating `REASSIGN OWNED BY` commands (#7319)
 * Adds support for propagating `CREATE`/`DROP` database from all nodes (#7240,
  #7253, #7359)
 * Propagates `SECURITY LABEL ON ROLE` statement from any node (#7508)
 * Adds support for issuing role management commands from worker nodes (#7278)
 * Adds support for propagating `ALTER USER RENAME` commands (#7204)
 * Adds support for propagating `ALTER DATABASE <db_name> SET ..` commands
  (#7181)
 * Adds support for propagating `SECURITY LABEL` on tables and columns (#7956)
 * Adds support for propagating `COMMENT ON <database>/<role>` commands (#7388)
 * Moves some of the internal citus functions from `pg_catalog` to
  `citus_internal` schema (#7473, #7470, #7466, 7456, 7450)
 * Adjusts `max_prepared_transactions` only when it's set to default on PG >= 16
  (#7712)
 * Adds skip_qualify_public param to shard_name() UDF to allow qualifying for
  "public" schema when needed (#8014)
 * Allows `citus_*_size` on indexes on a distributed tables (#7271)
 * Allows `GRANT ADMIN` to now also be `INHERIT` or `SET` in support of PG16
 * Makes sure `worker_copy_table_to_node` errors out with Citus tables (#7662)
 * Adds information to explain output when using
  `citus.explain_distributed_queries=false` (#7412)
 * Logs username in the failed connection message (#7432)
 * Makes sure to avoid incorrectly pushing-down the outer joins between
  distributed tables and recurring relations (like reference tables, local
  tables and `VALUES(..)` etc.) prior to PG 17 (#7937)
 * Prevents incorrectly pushing `nextval()` call down to workers to avoid using
  incorrect sequence value for some types of `INSERT .. SELECT`s (#7976)
 * Makes sure to prevent `INSERT INTO ... SELECT` queries involving subfield or
  sublink, to avoid crashes (#7912)
 * Makes sure to take improvement_threshold into the account
  in `citus_add_rebalance_strategy()` (#7247)
 * Makes sure to disallow creating a replicated distributed
  table concurrently (#7219)
 * Fixes a bug that causes omitting `CASCADE` clause for the commands sent to
  workers for `REVOKE` commands on tables (#7958)
 * Fixes an issue detected using address sanitizer (#7948, #7949)
 * Fixes a bug in deparsing of shard query in case of "output-table column" name
  conflict (#7932)
 * Fixes a crash in columnar custom scan that happens when a columnar table is
  used in a join (#7703)
 * Fixes `MERGE` command when insert value does not have source distributed
  column (#7627)
 * Fixes performance issue when using `\d tablename` on a server with many
  tables (#7577)
 * Fixes performance issue in `GetForeignKeyOids` on systems with many
  constraints (#7580)
 * Fixes performance issue when distributing a table that depends on an
  extension (#7574)
 * Fixes performance issue when creating distributed tables if many already
  exist (#7575)
 * Fixes a crash caused by some form of `ALTER TABLE ADD COLUMN` statements. When
  adding multiple columns, if one of the `ADD COLUMN` statements contains a
  `FOREIGN` constraint ommitting the referenced
  columns in the statement, a `SEGFAULT` occurs (#7522)
 * Fixes assertion failure in maintenance daemon during Citus upgrades  (#7537)
 * Fixes segmentation fault when using `CASE WHEN` in `DO` block functions
  (#7554)
 * Fixes undefined behavior in `master_disable_node` due to argument mismatch
  (#7492)
 * Fixes incorrect propagating of `GRANTED BY` and `CASCADE/RESTRICT` clauses
  for `REVOKE` statements (#7451)
 * Fixes the incorrect column count after `ALTER TABLE` (#7379)
 * Fixes timeout when underlying socket is changed for an inter-node connection
  (#7377)
 * Fixes memory leaks (#7441, #7440)
 * Fixes leaking of memory and memory contexts when tracking foreign keys between
  Citus tables (#7236)
 * Fixes a potential segfault for background rebalancer (#7694)
 * Fixes potential `NULL` dereference in casual clocks (#7704)
 ### citus v13.0.4 (May 29th, 2025) ###
 * Fixes an issue detected using address sanitizer (#7966)
 * Error out for queries with outer joins and pseudoconstant quals in versions
  prior to PG 17 (#7937)
 ### citus v12.1.8 (May 29, 2025) ###
 * Fixes a crash in left outer joins that can happen when there is an an
  aggregate on a column from the inner side of the join (#7904)
 * Fixes an issue detected using address sanitizer (#7965)
 * Fixes a crash when executing a prepared CALL, which is not pure SQL but
 available with some drivers like npgsql and jpgdbc (#7288)
 ### citus v13.0.3 (March 20th, 2025) ###
 * Fixes a version bump issue in 13.0.2
 ### citus v13.0.2 (March 12th, 2025) ###
 * Fixes a crash in columnar custom scan that happens when a columnar table is
  used in a join. (#7647)
 * Fixes a bug that breaks `UPDATE SET (...) = (SELECT some_func(),... )`
  type of queries on Citus tables (#7914)
 * Fixes a planning error caused by a redundant WHERE clause (#7907)
 * Fixes a crash in left outer joins that can happen when there is an aggregate
  on a column from the inner side of the join. (#7901)
 * Fixes deadlock with transaction recovery that is possible during Citus
  upgrades. (#7910)
 * Fixes a bug that prevents inserting into Citus tables that uses
  a GENERATED ALWAYS AS IDENTITY column. (#7920)
 * Ensures that a MERGE command on a distributed table with a WHEN NOT MATCHED BY
  SOURCE clause runs against all shards of the distributed table. (#7900)
 * Fixes a bug that breaks router updates on distributed tables
  when a reference table is used in the subquery (#7897)
 ### citus v12.1.7 (Feb 6, 2025) ###
 * Fixes a crash that happens because of unsafe catalog access when re-assigning
  the global pid after `application_name` changes (#7791)
 * Prevents crashes when another extension skips executing the
  `ClientAuthentication_hook` of Citus. (#7836)
 ### citus v13.0.1 (February 4th, 2025) ###
 * Drops support for PostgreSQL 14 (#7753)
 ### citus v13.0.0 (January 22, 2025) ###
 * Adds support for PostgreSQL 17 (#7699, #7661)
 * Adds `JSON_TABLE()` support in distributed queries (#7816)
 * Propagates `MERGE ... WHEN NOT MATCHED BY SOURCE` (#7807)
 * Propagates `MEMORY` and `SERIALIZE` options of `EXPLAIN` (#7802)
 * Adds support for identity columns in distributed partitioned tables (#7785)
 * Allows specifying an access method for distributed partitioned tables (#7818)
 * Allows exclusion constraints on distributed partitioned tables (#7733)
 * Allows configuring sslnegotiation using `citus.node_conn_info` (#7821)
 * Avoids wal receiver timeouts during large shard splits (#7229)
 * Fixes a bug causing incorrect writing of data to target `MERGE` repartition
  command (#7659)
 * Fixes a crash that happens because of unsafe catalog access when re-assigning
  the global pid after `application_name` changes (#7791)
 * Fixes incorrect `VALID UNTIL` setting assumption made for roles when syncing
  them to new nodes (#7534)
 * Fixes segfault when calling distributed procedure with a parameterized
  distribution argument (#7242)
 * Fixes server crash when trying to execute `activate_node_snapshot()` on a
  single-node cluster (#7552)
 * Improves `citus_move_shard_placement()` to fail early if there is a new node
  without reference tables yet (#7467)
 ### citus v12.1.6 (Nov 14, 2024) ###
 * Propagates `SECURITY LABEL .. ON ROLE` statements (#7304)
 * Fixes crash caused by running queries with window partition (#7718)
 ### citus v12.1.5 (July 17, 2024) ###
 * Adds support for MERGE commands with single shard distributed target tables
  (#7643)
 * Fixes an error with MERGE commands when insert value does not have source
  distribution column (#7627)
 ### citus v12.1.4 (May 28, 2024) ###
 * Adds null check for node in HasRangeTableRef (#7604)
 ### citus v12.1.3 (April 18, 2024) ###
 * Allows overwriting host name for all inter-node connections by
  supporting "host" parameter in citus.node_conninfo (#7541)
 * Avoids distributed deadlocks by changing the order in which the locks are
  acquired for the target and reference tables (#7542)
 * Fixes a performance issue when distributing a table that depends on an
  extension (#7574)
 * Fixes a performance issue when using "\d tablename" on a server with
  many tables (#7577)
 * Fixes a crash caused by some form of ALTER TABLE ADD COLUMN
  statements. When adding multiple columns, if one of the ADD COLUMN
  statements contains a FOREIGN constraint omitting the referenced
  columns in the statement, a SEGFAULT was occurring. (#7522)
 * Fixes a performance issue when creating distributed tables if many
  already exist (#7575, #7579)
 * Fixes a bug when hostname in pg_dist_node resolves to multiple IPs
  (#7377)
 * Fixes performance issue when tracking foreign key constraints on
  systems with many constraints (#7578)
 * Fixes segmentation fault when using CASE WHEN in DO block within
  functions. (#7554)
 * Fixes undefined behavior in master_disable_node due to argument
  mismatch (#7492)
 * Fixes some potential bugs by correctly marking some variables as
  volatile (#7570)
 * Logs username in the failed connection message (#7432)
 ### citus v11.0.10 (February 15, 2024) ###
 * Removes pg_send_cancellation and all references (#7135)
 ### citus v12.1.2 (February 12, 2024) ###
 * Fixes the incorrect column count after ALTER TABLE (#7379)
 ### citus v12.0.1 (July 11, 2023) ###
 * Fixes incorrect default value assumption for VACUUM(PROCESS_TOAST) #7122)
 * Fixes a bug that causes an unexpected error when adding a column
  with a NULL constraint (#7093)
 * Fixes a bug that could cause COPY logic to skip data in case of OOM (#7152)
 * Fixes a bug with deleting colocation groups (#6929)
 * Fixes memory and memory contexts leaks in Foreign Constraint Graphs (#7236)
 * Fixes shard size bug with too many shards (#7018)
 * Fixes the incorrect column count after ALTER TABLE (#7379)
 * Improves citus_tables view performance (#7050)
 * Makes sure to disallow creating a replicated distributed table
  concurrently (#7219)
 * Removes pg_send_cancellation and all references (#7135)
 ### citus v11.3.1 (February 12, 2024) ###
 * Disallows MERGE when the query prunes down to zero shards (#6946)
 * Fixes a bug related to non-existent objects in DDL commands (#6984)
 * Fixes a bug that could cause COPY logic to skip data in case of OOM (#7152)
 * Fixes a bug with deleting colocation groups (#6929)
 * Fixes incorrect results on fetching scrollable with hold cursors (#7014)
 * Fixes memory and memory context leaks in Foreign Constraint Graphs (#7236)
 * Fixes replicate reference tables task fail when user is superuser (#6930)
 * Fixes the incorrect column count after ALTER TABLE (#7379)
 * Improves citus_shard_sizes performance (#7050)
 * Makes sure to disallow creating a replicated distributed table
  concurrently (#7219)
 * Removes pg_send_cancellation and all references (#7135)
 ### citus v11.2.2 (February 12, 2024) ###
 * Fixes a bug in background shard rebalancer where the replicate
  reference tables task fails if the current user is not a superuser (#6930)
 * Fixes a bug related to non-existent objects in DDL commands (#6984)
 * Fixes a bug that could cause COPY logic to skip data in case of OOM (#7152)
 * Fixes a bug with deleting colocation groups (#6929)
 * Fixes incorrect results on fetching scrollable with hold cursors (#7014)
 * Fixes memory and memory context leaks in Foreign Constraint Graphs (#7236)
 * Fixes the incorrect column count after ALTER TABLE (#7379)
 * Improves failure handling of distributed execution (#7090)
 * Makes sure to disallow creating a replicated distributed table
  concurrently (#7219)
 * Removes pg_send_cancellation (#7135)
 ### citus v11.1.7 (February 12, 2024) ###
 * Fixes memory and memory context leaks in Foreign Constraint Graphs (#7236)
 * Fixes a bug related to non-existent objects in DDL commands (#6984)
 * Fixes a bug that could cause COPY logic to skip data in case of OOM (#7152)
 * Fixes a bug with deleting colocation groups (#6929)
 * Fixes incorrect results on fetching scrollable with hold cursors (#7014)
 * Fixes the incorrect column count after ALTER TABLE (#7379)
 * Improves failure handling of distributed execution (#7090)
 * Makes sure to disallow creating a replicated distributed table
  concurrently (#7219)
 * Removes pg_send_cancellation and all references (#7135)
 ### citus v11.0.9 (February 12, 2024) ###
 * Fixes a bug that could cause COPY logic to skip data in case of OOM (#7152)
 * Fixes a bug with deleting colocation groups (#6929)
 * Fixes memory and memory context leaks in Foreign Constraint Graphs (#7236)
 * Fixes the incorrect column count after ALTER TABLE (#7462)
 * Improve failure handling of distributed execution (#7090)
 ### citus v12.1.1 (November 9, 2023) ###
 * Fixes leaking of memory and memory contexts in Citus foreign key cache
  (#7236)
 * Makes sure to disallow creating a replicated distributed table concurrently
  (#7219)
 ### citus v12.1.0 (September 12, 2023) ###
 * Adds support for PostgreSQL 16.0 (#7173)
 * Add `citus_schema_move()` function which moves tables within a
  distributed schema to another node (#7180)
 * Adds `citus_pause_node_within_txn()` UDF that allows pausing the node with
  given id (#7089)
 * Makes sure to enforce shard level colocation with the GUC
  `citus.enable_non_colocated_router_query_pushdown` (#7076)
 * Allows creating reference / distributed-schema tables from local tables added
  to metadata and that use identity columns (#7131)
 * Propagates `BUFFER_USAGE_LIMIT` option in `VACUUM` and `ANALYZE` (#7114)
 * Propagates `PROCESS_MAIN`, `SKIP_DATABASE_STATS`, `ONLY_DATABASE_STATS`
  options in `VACUUM` (#7114)
 * Propagates `GENERIC_PLAN` option in `EXPLAIN` (#7141)
 * Propagates "rules" option in `CREATE COLLATION` (#7185)
 * Propagates `GRANT`/ `REVOKE` for database privileges (#7109)
 * Adds TRUNCATE trigger support on Citus foreign tables (#7170)
 * Removes `pg_send_cancellation` (#7135)
 * Prevents unnecessarily pulling the data into coordinator for some
  `INSERT .. SELECT` queries that target a single-shard group (#7077)
 * Makes sure that rebalancer throws an error if replication factor is greater
  than the shard allowed node count. Also makes sure to avoid moving a shard
  to a node that it already exists on. (#7074)
 * Fixes a bug that may appear during 2PC recovery when there are multiple
  databases (#7174)
 * Fixes a bug that could cause `COPY` logic to skip data in case of
  out-of-memory (#7152)
 * Fixes a bug that causes an unexpected error when adding a column with
  a `NULL` constraint (#7093)
 * Fixes `PROCESS_TOAST` default value to `true` (#7122)
 * Improves the error thrown when there is datatype mismatch in `MERGE ON`
  (#7081)
 ### citus v12.0.0 (July 11, 2023) ###
 * Adds support for schema-based sharding.
  While `citus.enable_schema_based_sharding` GUC allows sharding the database
  based on newly created schemas, `citus_schema_distribute()` allows doing so
  for the existing schemas. Distributed schemas used for sharding the database
  can be listed by using the view `citus_schemas`, monitored by using the view
  `citus_stat_schemas`, and undistributed by using the udf
  `citus_schema_undistribute()`
  (#6866, #6979, #6933, #6936 and many others)
 * Supports MERGE command across non-colocated distributed tables/subqueries,
  reference tables and joins on non-distribution columns (#6927)
 * Drops PG13 Support (#7002, #7007)
 * Changes default rebalance strategy to by_disk_size (#7033)
 * Changes by_disk_size rebalance strategy to have a base size (#7035)
 * Improves citus_tables view performance (#7018)
 * Improves tenant monitoring performance (#6868)
 * Introduces the GUC `citus.stat_tenants_untracked_sample_rate` for sampling in
  tenant monitoring (#7026)
 * Adds CPU usage to citus_stat_tenants (#6844)
 * Propagates `ALTER SCHEMA .. OWNER TO ..` commands to worker (#6987)
 * Allows `ADD COLUMN` in command string with other commands (#7032)
 * Allows `DROP CONSTRAINT` in command string with other commands (#7012)
 * Makes sure to properly handle index storage options for `ADD CONSTRAINT
  `/ COLUMN commands (#7032)
 * Makes sure to properly handle `IF NOT EXISTS` for `ADD COLUMN` commands
  (#7032)
 * Allows using generated identity column based on int/smallint when creating
  a distributed table with the limitation of not being able perform DMLs on
  identity columns from worker nodes (#7008)
 * Supports custom cast from / to timestamptz in time partition management UDFs
  (#6923)
 * Optimizes pushdown planner on memory and cpu (#6945)
 * Changes citus_shard_sizes view's table_name column to shard_id (#7003)
 * The GUC search_path is now reported when it is updated (#6983)
 * Disables citus.enable_non_colocated_router_query_pushdown GUC by default to
  ensure generating a consistent distributed plan for the queries that
  reference non-colocated distributed tables (#6909)
 * Disallows MERGE with filters that prune down to zero shards (#6946)
 * Makes sure to take `shouldhaveshards` setting into account for a node when
  planning rebalance steps (#6887)
 * Improves the compatibility with other extension by forwarding to existing
  emit_log_hook in our log hook (#6877)
 * Fixes wrong result when using `NOT MATCHED` with MERGE command (#6943)
 * Fixes querying the view `citus_shard_sizes` when there are too many shards
  (#7018)
 * Fixes a bug related to type casts from other types to text/varchar (#6391)
 * Fixes propagating `CREATE SCHEMA AUTHORIZATION ..` with no schema name
  (#7015)
 * Fixes an error when creating a FOREIGN KEY without a name referencing a schema
  qualified table (#6986)
 * Fixes a rare bug which mostly happens with queries that contain both outer
  join and where clauses  (#6857)
 * Fixes a bug related to propagation of schemas when pg_dist_node is empty
  (#6900)
 * Fixes a crash when a query is locally executed with explain analyze (#6892)
 ### citus v11.3.0 (May 2, 2023) ###
 * Introduces CDC implementation for Citus using logical replication
  (#6623, #6810, #6827)
 * Adds support for `MERGE` command on co-located distributed tables joined on
  distribution column (#6696, #6733)
 * Adds the view `citus_stat_tenants` that monitor statistics on tenant usages
  (#6725)
 * Adds the GUC `citus.max_background_task_executors_per_node` to control number
  of background task executors involving a node (#6771)
 * Allows parallel shard moves in background rebalancer (#6756)
 * Introduces the GUC `citus.metadata_sync_mode` that introduces nontransactional
  mode for metadata sync (#6728, #6889)
 * Propagates CREATE/ALTER/DROP PUBLICATION statements for distributed tables
  (#6776)
 * Adds the GUC `citus.enable_non_colocated_router_query_pushdown` to ensure
  generating a consistent distributed plan for the queries that reference
  non-colocated distributed tables when set to "false" (#6793)
 * Checks if all moves are able to be done via logical replication for rebalancer
  (#6754)
 * Correctly reports shard size in `citus_shards` view (#6748)
 * Fixes a bug in shard copy operations (#6721)
 * Fixes a bug that prevents enforcing identity column restrictions on worker
  nodes (#6738)
 * Fixes a bug with `INSERT .. SELECT` queries with identity columns (#6802)
 * Fixes an issue that caused some queries with custom aggregates to fail (#6805)
 * Fixes an issue when `citus_set_coordinator_host` is called more than once
  (#6837)
 * Fixes an uninitialized memory access in shard split API (#6845)
 * Fixes memory leak and max allocation block errors during metadata syncing
  (#6728)
 * Fixes memory leak in `undistribute_table` (#6693)
 * Fixes memory leak in `alter_distributed_table` (#6726)
 * Fixes memory leak in `create_distributed_table` (#6722)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Improves rebalancer when shard groups have placement count less than worker
  count (#6739)
 * Makes sure to stop maintenance daemon when dropping a database even without
  Citus extension (#6688)
 * Prevents using `alter_distributed_table` and `undistribute_table` UDFs when a
  table has identity columns (#6738)
 * Prevents using identity columns on data types other than `bigint` on
  distributed tables (#6738)
 ### citus v11.2.1 (April 20, 2023) ###
 * Correctly reports shard size in `citus_shards` view (#6748)
 * Fixes a bug in shard copy operations (#6721)
 * Fixes a bug with `INSERT .. SELECT` queries with identity columns (#6802)
 * Fixes an uninitialized memory access in shard split API (#6845)
 * Fixes compilation for PG13.10 and PG14.7 (#6711)
 * Fixes memory leak in `alter_distributed_table` (#6726)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Prevents using `alter_distributed_table` and `undistribute_table` UDFs when a
  table has identity columns (#6738)
 * Prevents using identity columns on data types other than `bigint` on
  distributed tables (#6738)
 ### citus v11.1.6 (April 20, 2023) ###
 * Correctly reports shard size in `citus_shards` view (#6748)
 * Fixes a bug in shard copy operations (#6721)
 * Fixes a bug that breaks pg upgrades if the user has a columnar table (#6624)
 * Fixes a bug that causes background rebalancer to fail when a reference table
  doesn't have a primary key (#6682)
 * Fixes a regression in allowed foreign keys on distributed tables (#6550)
 * Fixes a use-after-free bug in connection management (#6685)
 * Fixes an unexpected foreign table error by disallowing to drop the
  `table_name` option (#6669)
 * Fixes an uninitialized memory access in shard split API (#6845)
 * Fixes compilation for PG13.10 and PG14.7 (#6711)
 * Fixes crash that happens when trying to replicate a reference table that is
  actually dropped (#6595)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Fixes the modifiers for subscription and role creation (#6603)
 * Makes sure to quote all identifiers used for logical replication to prevent
  potential issues (#6604)
 * Makes sure to skip foreign key validations at the end of shard moves (#6640)
 ### citus v11.0.8 (April 20, 2023) ###
 * Correctly reports shard size in `citus_shards` view (#6748)
 * Fixes a bug that breaks pg upgrades if the user has a columnar table (#6624)
 * Fixes an unexpected foreign table error by disallowing to drop the
  `table_name` option (#6669)
 * Fixes compilation warning on PG13 + OpenSSL 3.0 (#6038, #6502)
 * Fixes crash that happens when trying to replicate a reference table that is
  actually dropped (#6595)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Fixes the modifiers for subscription and role creation (#6603)
 * Fixes two potential dangling pointer issues (#6504, #6507)
 * Makes sure to quote all identifiers used for logical replication to prevent
  potential issues (#6604)
 ### citus v10.2.9 (April 20, 2023) ###
 * Correctly reports shard size in `citus_shards` view (#6748)
 * Fixes a bug in `ALTER EXTENSION citus UPDATE` (#6383)
 * Fixes a bug that breaks pg upgrades if the user has a columnar table (#6624)
 * Fixes a bug that prevents retaining columnar table options after a
  table-rewrite (#6337)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Raises memory limits in columnar from 256MB to 1GB for reads and writes
  (#6419)
 ### citus v10.1.6 (April 20, 2023) ###
 * Fixes a crash that occurs when the aggregate that cannot be pushed-down
  returns empty result from a worker (#5679)
 * Fixes columnar freezing/wraparound bug (#5962)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Prevents alter table functions from dropping extensions (#5974)
 ### citus v10.0.8 (April 20, 2023) ###
 * Fixes a bug that could break `DROP SCHEMA/EXTENSON` commands when there is a
  columnar table (#5458)
 * Fixes a crash that occurs when the aggregate that cannot be pushed-down
  returns empty result from a worker (#5679)
 * Fixes columnar freezing/wraparound bug (#5962)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Prevents alter table functions from dropping extensions (#5974)
 ### citus v9.5.12 (April 20, 2023) ###
 * Fixes a crash that occurs when the aggregate that cannot be pushed-down
  returns empty result from a worker (#5679)
 * Fixes memory leak issue with query results that returns single row (#6724)
 * Prevents alter table functions from dropping extensions (#5974)
 ### citus v11.2.0 (January 30, 2023) ###
 * Adds support for outer joins with reference tables / complex subquery-CTEs
  in the outer side of the join (e.g., \<reference table\> LEFT JOIN
  \<distributed table\>)
 * Adds support for creating `PRIMARY KEY`s and `UNIQUE`/`EXCLUSION`/`CHECK`/
  `FOREIGN KEY` constraints via `ALTER TABLE` command without providing a
  constraint name
 * Adds support for using identity columns on Citus tables
 * Adds support for `MERGE` command on local tables
 * Adds `citus_job_list()`, `citus_job_status()` and `citus_rebalance_status()`
  UDFs that allow monitoring rebalancer progress
 * Adds `citus_task_wait()` UDF to wait on desired task status
 * Adds `source_lsn`, `target_lsn` and `status` fields into
  `get_rebalance_progress()`
 * Introduces `citus_copy_shard_placement()` UDF with node id
 * Introduces `citus_move_shard_placement()` UDF with node id
 * Propagates `BEGIN` properties to worker nodes
 * Propagates `DROP OWNED BY` to worker nodes
 * Deprecates `citus.replicate_reference_tables_on_activate` and makes it
  always `off`
 * Drops GUC `citus.defer_drop_after_shard_move`
 * Drops GUC `citus.defer_drop_after_shard_split`
 * Drops `SHARD_STATE_TO_DELETE` state and uses the cleanup records instead
 * Allows `citus_update_node()` to work with nodes from different clusters
 * Adds signal handlers for queue monitor to gracefully shutdown, cancel and to
  see config changes
 * Defers cleanup after a failure in shard move or split
 * Extends cleanup process for replication artifacts
 * Improves a query that terminates compelling backends from
  `citus_update_node()`
 * Includes Citus global pid in all internal `application_name`s
 * Avoids leaking `search_path` to workers when executing DDL commands
 * Fixes `alter_table_set_access_method error()` for views
 * Fixes `citus_drain_node()` to allow draining the specified worker only
 * Fixes a bug in global pid assignment for connections opened by rebalancer
  internally
 * Fixes a bug that causes background rebalancer to fail when a reference table
  doesn't have a primary key
 * Fixes a bug that might cause failing to query the views based on tables that
  have renamed columns
 * Fixes a bug that might cause incorrectly planning the sublinks in query tree
 * Fixes a floating point exception during
  `create_distributed_table_concurrently()`
 * Fixes a rebalancer failure due to integer overflow in subscription and role
  creation
 * Fixes a regression in allowed foreign keys on distributed tables
 * Fixes a use-after-free bug in connection management
 * Fixes an unexpected foreign table error by disallowing to drop the
  table_name option
 * Fixes an uninitialized memory access in `create_distributed_function()`
 * Fixes crash that happens when trying to replicate a reference table that is
  actually dropped
 * Make sure to cleanup the shard on the target node in case of a
  failed/aborted shard move
 * Makes sure to create replication artifacts with unique names
 * Makes sure to disallow triggers that depend on extensions
 * Makes sure to quote all identifiers used for logical replication to prevent
  potential issues
 * Makes sure to skip foreign key validations at the end of shard moves
 * Prevents crashes on `UPDATE` with certain `RETURNING` clauses
 * Propagates column aliases in the shard-level commands
 ### citus v11.1.5 (December 12, 2022) ###
 * Fixes two potential dangling pointer issues
 * Fixes compilation warning on PG13 + OpenSSL 3.0
 ### citus v11.0.7 (November 8, 2022) ###
 * Adds the GUC `citus.allow_unsafe_constraints` to allow unique/exclusion/
  primary key constraints without distribution column
 * Allows `citus_internal` `application_name` with additional suffix
 * Disallows having `ON DELETE/UPDATE SET DEFAULT` actions on columns that
  default to sequences
 * Fixes a bug in `ALTER EXTENSION citus UPDATE`
 * Fixes a bug that causes a crash with empty/null password
 * Fixes a bug that causes not retaining trigger enable/disable settings when
  re-creating them on shards
 * Fixes a bug that might cause inserting incorrect `DEFAULT` values when
  applying foreign key actions
 * Fixes a bug that prevents retaining columnar table options after a
  table-rewrite
 * Fixes a bug that prevents setting colocation group of a partitioned
  distributed table to `none`
 * Fixes an issue that can cause logical reference table replication to fail
 * Raises memory limits in columnar from 256MB to 1GB for reads and writes
 ### citus v11.1.4 (October 24, 2022) ###
 * Fixes an upgrade problem for `worker_fetch_foreign_file` when upgrade path
  starts from 8.3 up to 11.1
 * Fixes an upgrade problem for `worker_repartition_cleanup` when upgrade path
  starts from 9.1 up to 11.1
 ### citus v11.1.3 (October 14, 2022) ###
 * Adds support for PostgreSQL 15.0
 * Fixes a bug in `ALTER EXTENSION citus UPDATE`
 * Fixes a bug that causes a crash with empty/null password
 * Fixes a bug that causes not retaining trigger enable/disable settings when
  re-creating them on shards
 * Fixes a bug that prevents retaining columnar table options after a
  table-rewrite
 * Raises memory limits in columnar from 256MB to 1GB for reads and writes
 ### citus v11.1.2 (September 30, 2022) ###
 * Adds support for PostgreSQL 15rc1
 * Disallows having `ON DELETE/UPDATE SET DEFAULT` actions on columns that
  default to sequences
--- a/CODE_OF_CONDUCT.md
+++ b/CODE_OF_CONDUCT.md
@ -1,9 +0,0 @@
 # Microsoft Open Source Code of Conduct
 This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
 Resources:
 - [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
 - [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/)
 - Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -11,52 +11,6 @@ sign a Contributor License Agreement (CLA). For an explanation of
 why we ask this as well as instructions for how to proceed, see the
 [Microsoft CLA](https://cla.opensource.microsoft.com/).
 ### Devcontainer / Github Codespaces
 The easiest way to start contributing is via our devcontainer. This container works both locally in visual studio code with docker-desktop/docker-for-mac as well as [Github Codespaces](https://github.com/features/codespaces). To open the project in vscode you will need the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers). For codespaces you will need to [create a new codespace](https://codespace.new/citusdata/citus).
 With the extension installed you can run the following from the command pallet to get started
 ```
 > Dev Containers: Clone Repository in Container Volume...
 ```
 In the subsequent popup paste the url to the repo and hit enter.
 ```
 https://github.com/citusdata/citus
 ```
 This will create an isolated Workspace in vscode, complete with all tools required to build, test and run the Citus extension. We keep this container up to date with the supported postgres versions as well as the exact versions of tooling we use.
 To quickly start we suggest splitting your terminal once to have two shells. The left one in the `/workspaces/citus`, the second one changed to `/data`. The left terminal will be used to interact with the project, the right one with a testing cluster.
 To get citus installed from source we run `make install -s` in the first terminal. Once installed you can start a Citus cluster in the second terminal via `citus_dev make citus`. The cluster will run in the background, and can be interacted with via `citus_dev`. To get an overview of the available commands.
 With the Citus cluster running you can connect to the coordinator in the first terminal via `psql -p9700`. Because the coordinator is the most common entrypoint the `PGPORT` environment is set accordingly, so a simple `psql` will connect directly to the coordinator.
 ### Debugging in the VS code
 1. Start Debugging: Press F5 in VS Code to start debugging. When prompted, you'll need to attach the debugger to the appropriate PostgreSQL process.
 2. Identify the Process: If you're running a psql command, take note of the PID that appears in your psql prompt. For example:
 ```
 [local] citus@citus:9700 (PID: 5436)=#
 ```
 This PID (5436 in this case) indicates the process that you should attach the debugger to.
 If you are uncertain about which process to attach, you can list all running PostgreSQL processes using the following command:
 ```
 ps aux | grep postgres
 ```
 Look for the process associated with the PID you noted. For example:
 ```
 citus      5436  0.0  0.0  0  0 ?        S    14:00   0:00 postgres: citus citus
 ```
 4. Attach the Debugger: Once you've identified the correct PID, select that process when prompted in VS Code to attach the debugger. You should now be able to debug the PostgreSQL session tied to the psql command.
 5. Set Breakpoints and Debug: With the debugger attached, you can set breakpoints within the code. This allows you to step through the code execution, inspect variables, and fully debug the PostgreSQL instance running in your container.
 ### Getting and building
 [PostgreSQL documentation](https://www.postgresql.org/support/versioning/) has a
@ -87,8 +41,6 @@ that are missing in earlier minor versions.
  cd citus
  ./configure
  # If you have already installed the project, you need to clean it first
  make clean
  make
  make install
  # Optionally, you might instead want to use `make install-all`
@ -127,8 +79,6 @@ that are missing in earlier minor versions.
  git clone https://github.com/citusdata/citus.git
  cd citus
  ./configure
  # If you have already installed the project previously, you need to clean it first
  make clean
  make
  sudo make install
  # Optionally, you might instead want to use `sudo make install-all`
@ -179,8 +129,6 @@ that are missing in earlier minor versions.
  git clone https://github.com/citusdata/citus.git
  cd citus
  PG_CONFIG=/usr/pgsql-14/bin/pg_config ./configure
  # If you have already installed the project previously, you need to clean it first
  make clean
  make
  sudo make install
  # Optionally, you might instead want to use `sudo make install-all`
@ -197,7 +145,43 @@ that are missing in earlier minor versions.
 ### Following our coding conventions
-Our coding conventions are documented in [STYLEGUIDE.md](STYLEGUIDE.md).
+CircleCI will automatically reject any PRs which do not follow our coding
 conventions. The easiest way to ensure your PR adheres to those conventions is
 to use the [citus_indent](https://github.com/citusdata/tools/tree/develop/uncrustify)
 tool. This tool uses `uncrustify` under the hood.
 ```bash
 # Uncrustify changes the way it formats code every release a bit. To make sure
 # everyone formats consistently we use version 0.68.1:
 curl -L https://github.com/uncrustify/uncrustify/archive/uncrustify-0.68.1.tar.gz | tar xz
 cd uncrustify-uncrustify-0.68.1/
 mkdir build
 cd build
 cmake ..
 make -j5
 sudo make install
 cd ../..
 git clone https://github.com/citusdata/tools.git
 cd tools
 make uncrustify/.install
 ```
 Once you've done that, you can run the `make reindent` command from the top
 directory to recursively check and correct the style of any source files in the
 current directory. Under the hood, `make reindent` will run `citus_indent` and
 some other style corrections for you.
 You can also run the following in the directory of this repository to
 automatically format all the files that you have changed before committing:
 ```bash
 cat > .git/hooks/pre-commit << __EOF__
 #!/bin/bash
 citus_indent --check --diff || { citus_indent --diff; exit 1; }
 __EOF__
 chmod +x .git/hooks/pre-commit
 ```
 ### Making SQL changes
@ -234,50 +218,3 @@ style `#include` statements like this:
 Any other SQL you can put directly in the main sql file, e.g.
 `src/backend/distributed/sql/citus--8.3-1--9.0-1.sql`.
 ### Backporting a commit to a release branch
 1. Check out the release branch that you want to backport to `git checkout release-11.3`
 2. Make sure you have the latest changes `git pull`
 3. Create a new release branch with a unique name `git checkout -b release-11.3-<yourname>`
 4. Cherry-pick the commit that you want to backport `git cherry-pick -x <sha>` (the `-x` is important)
 5. Push the branch `git push`
 6. Wait for tests to pass
 7. If the cherry-pick required non-trivial merge conflicts, create a PR and ask
   for a review.
 8. After the tests pass on CI, fast-forward the release branch `git push origin release-11.3-<yourname>:release-11.3`
 ### Running tests
 See [`src/test/regress/README.md`](https://github.com/citusdata/citus/blob/master/src/test/regress/README.md)
 ### Documentation
 User-facing documentation is published on [docs.citusdata.com](https://docs.citusdata.com/). When adding a new feature, function, or setting, you can open a pull request or issue against the [Citus docs repo](https://github.com/citusdata/citus_docs/).
 Detailed descriptions of the implementation for Citus developers are provided in the [Citus Technical Documentation](src/backend/distributed/README.md). It is currently a single file for ease of searching. Please update the documentation if you make any changes that affect the design or add major new features.
 # Making a pull request ready for reviews
 Asking for help and asking for reviews are two different things. When you're asking for help, you're asking for someone to help you with something that you're not expected to know.
 But when you're asking for a review, you're asking for someone to review your work and provide feedback. So, when you're asking for a review, you're expected to make sure that:
 * Your changes don't perform **unnecessary line addition / deletions / style changes on unrelated files / lines**.
 * All CI jobs are **passing**, including **style checks** and **flaky test detection jobs**. Note that if you're an external contributor, you don't have to wait CI jobs to run (and finish) because they don't get automatically triggered for external contributors.
 * Your PR has necessary amount of **tests** and that they're passing.
 * You separated as much as possible work into **separate PRs**, e.g., a prerequisite bugfix, a refactoring etc..
 * Your PR doesn't introduce a typo or something that you can easily fix yourself.
 * After all CI jobs pass, code-coverage measurement job (CodeCov as of today) then kicks in. That's why it's important to make the **tests passing** first. At that point, you're expected to check **CodeCov annotations** that can be seen in the **Files Changed** tab and expected to make sure that it doesn't complain about any lines that are not covered. For example, it's ok if CodeCov complains about an `ereport()` call that you put for an "unexpected-but-better-than-crashing" case, but it's not ok if it complains about an uncovered `if` branch that you added.
 * And finally, perform a **self-review** to make sure that:
  * Code and code-comments reflects the idea **without requiring an extra explanation** via a chat message / email / PR comment.
    This is important because we don't expect developers to reach out to author / read about the whole discussion in the PR to understand the idea behind a commit merged into `main` branch.
  * PR description is clear enough.
  * If-and-only-if you're **introducing a user facing change / bugfix**, your PR has a line that starts with `DESCRIPTION: <Present simple tense word that starts with a capital letter, e.g., Adds support for / Fixes / Disallows>`.
  * **Commit messages** are clear enough if the commits are doing logically different things.
--- a/DEVCONTAINER.md
+++ b/DEVCONTAINER.md
@ -1,43 +0,0 @@
 # Devcontainer
 ## Coredumps
 When postgres/citus crashes, there is the option to create a coredump. This is useful for debugging the issue. Coredumps are enabled in the devcontainer by default. However, not all environments are configured correctly out of the box. The most important configuration that is not standardized is the `core_pattern`. The configuration can be verified from the container, however, you cannot change this setting from inside the container as the filesystem containing this setting is in read only mode while inside the container.
 To verify if corefiles are written run the following command in a terminal. This shows the filename pattern with which the corefile will be written.
 ```bash
 cat /proc/sys/kernel/core_pattern
 ```
 This should be configured with a relative path or simply a simple filename, such as `core`. When your environment shows an absolute path you will need to change this setting. How to change this setting depends highly on the underlying system as the setting needs to be changed on the kernel of the host running the container.
 You can put any pattern in `/proc/sys/kernel/core_pattern` as you see fit. eg. You can add the PID to the core pattern in one of two ways;
 - You either include `%p` in the core_pattern. This gets substituted with the PID of the crashing process.
 - Alternatively you could set `/proc/sys/kernel/core_uses_pid` to `1` in the same way as you set `core_pattern`. This will append the PID to the corefile if `%p` is not explicitly contained in the core_pattern.
 When a coredump is written you can use the debug/launch configuration `Open core file` which is preconfigured in the devcontainer. This will open a fileprompt that lists all coredumps that are found in your workspace. When you want to debug coredumps from `citus_dev` that are run in your `/data` directory, you can add the data directory to your workspace. In the command pallet of vscode you can run `>Workspace: Add Folder to Workspace...` and select the `/data` directory. This will allow you to open the coredumps from the `/data` directory in the `Open core file` debug configuration.
 ### Windows (docker desktop)
 When running in docker desktop on windows you will most likely need to change this setting. The linux guest in WSL2 that runs your container is the `docker-desktop` environment. The easiest way to get onto the host, where you can change this setting, is to open a powershell window and verify you have the docker-desktop environment listed.
 ```powershell
 wsl --list
 ```
 Among others this should list both `docker-desktop` and `docker-desktop-data`. You can then open a shell in the `docker-desktop` environment.
 ```powershell
 wsl -d docker-desktop
 ```
 Inside this shell you can verify that you have the right environment by running
 ```bash
 cat /proc/sys/kernel/core_pattern
 ```
 This should show the same configuration as the one you see inside the devcontainer. You can then change the setting by running the following command.
 This will change the setting for the current session. If you want to make the change permanent you will need to add this to a startup script.
 ```bash
 echo "core" > /proc/sys/kernel/core_pattern
 ```
--- a/22
+++ b/22
@ -11,7 +11,7 @@ endif
 include Makefile.global
-all: extension
+all: extension pg_send_cancellation
 # build columnar only
@ -40,28 +40,32 @@ clean-full:
 install-downgrades:
 	$(MAKE) -C src/backend/distributed/ install-downgrades
-install-all: install-headers
+install-all: install-headers install-pg_send_cancellation
 	$(MAKE) -C src/backend/columnar/ install-all
 	$(MAKE) -C src/backend/distributed/ install-all
 # build citus_send_cancellation binary
 pg_send_cancellation:
 	$(MAKE) -C src/bin/pg_send_cancellation/ all
 install-pg_send_cancellation: pg_send_cancellation
 	$(MAKE) -C src/bin/pg_send_cancellation/ install
 clean-pg_send_cancellation:
 	$(MAKE) -C src/bin/pg_send_cancellation/ clean
 .PHONY: pg_send_cancellation install-pg_send_cancellation clean-pg_send_cancellation
 # Add to generic targets
-install: install-extension install-headers
+install: install-extension install-headers install-pg_send_cancellation
-clean: clean-extension
+clean: clean-extension clean-pg_send_cancellation
 # apply or check style
 reindent:
 	${citus_abs_top_srcdir}/ci/fix_style.sh
 check-style:
 	black . --check --quiet
 	isort . --check --quiet
 	flake8
 	cd ${citus_abs_top_srcdir} && citus_indent --quiet --check
 .PHONY: reindent check-style
 # depend on install-all so that downgrade scripts are installed as well
 check: all install-all
-	# explicetely does not use $(MAKE) to avoid parallelism
+	$(MAKE) -C src/test/regress check-full
 	make -C src/test/regress check
 .PHONY: all check clean install install-downgrades install-all
--- a/README.md
+++ b/README.md
@ -1,14 +1,14 @@
-| **<br/>The Citus database is 100% open source.<br/><img width=1000/><br/>Learn what's new in the [Citus 13.0 release blog](https://www.citusdata.com/blog/2025/02/06/distribute-postgresql-17-with-citus-13/) and the [Citus Updates page](https://www.citusdata.com/updates/).<br/><br/>**|
+| **<br/>Citus is now 100% open source and supports querying from any node.<br/><img width=1000/><br/>Read about it on the [Citus 11.0 release blog](https://www.citusdata.com/blog/2022/06/17/citus-11-goes-fully-open-source/) and the [Citus Updates page](https://www.citusdata.com/updates/).<br/><br/>** |
 |---|
 <br/>
 <br/>
-
+![Citus Banner](/citus-readme-banner.png)
 ![Citus Banner](images/citus-readme-banner.png)
 [![Latest Docs](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://docs.citusdata.com/)
 [![Stack Overflow](https://img.shields.io/badge/Stack%20Overflow-%20-545353?logo=Stack%20Overflow)](https://stackoverflow.com/questions/tagged/citus)
-[![Slack](https://cituscdn.azureedge.net/images/social/slack-badge.svg)](https://slack.citusdata.com/)
+[![Slack Status](https://citus-slack.herokuapp.com/badge.svg)](https://citus-public.slack.com/)
 [![Code Coverage](https://codecov.io/gh/citusdata/citus/branch/master/graph/badge.svg)](https://app.codecov.io/gh/citusdata/citus)
 [![Twitter](https://img.shields.io/twitter/follow/citusdata.svg?label=Follow%20@citusdata)](https://twitter.com/intent/follow?screen_name=citusdata)
@ -31,15 +31,13 @@ You can use these Citus superpowers to make your Postgres database scale-out rea
 Our [SIGMOD '21](https://2021.sigmod.org/) paper [Citus: Distributed PostgreSQL for Data-Intensive Applications](https://doi.org/10.1145/3448016.3457551) gives a more detailed look into what Citus is, how it works, and why it works that way.
-![Citus scales out from a single node](images/citus-scale-out.png)
+![Citus scales out from a single node](/citus-scale-out.png)
 Since Citus is an extension to Postgres, you can use Citus with the latest Postgres versions. And Citus works seamlessly with the PostgreSQL tools and extensions you are already familiar with.
 - [Why Citus?](#why-citus)
 - [Getting Started](#getting-started)
 - [Using Citus](#using-citus)
 - [Schema-based sharding](#schema-based-sharding)
 - [Setting up with High Availability](#setting-up-with-high-availability)
 - [Documentation](#documentation)
 - [Architecture](#architecture)
 - [When to Use Citus](#when-to-use-citus)
@ -65,11 +63,11 @@ Developers choose Citus for two reasons:
 ## Getting Started
-The quickest way to get started with Citus is to use the [Azure Cosmos DB for PostgreSQL](https://learn.microsoft.com/azure/cosmos-db/postgresql/quickstart-create-portal) managed service in the cloud—or [set up Citus locally](https://docs.citusdata.com/en/stable/installation/single_node.html).
+The quickest way to get started with Citus is to use the [Hyperscale (Citus)](https://docs.microsoft.com/azure/postgresql/quickstart-create-hyperscale-portal) deployment option in the Azure Database for PostgreSQL managed service—or [set up Citus locally](https://docs.citusdata.com/en/stable/installation/single_node.html).
-### Citus Managed Service on Azure
+### Hyperscale (Citus) on Azure Database for PostgreSQL
-You can get a fully-managed Citus cluster in minutes through the [Azure Cosmos DB for PostgreSQL portal](https://azure.microsoft.com/products/cosmos-db/). Azure will manage your backups, high availability through auto-failover, software updates, monitoring, and more for all of your servers. To get started Citus on Azure, use the [Azure Cosmos DB for PostgreSQL Quickstart](https://learn.microsoft.com/azure/cosmos-db/postgresql/quickstart-create-portal).
+You can get a fully-managed Citus cluster in minutes through the Hyperscale (Citus) deployment option in the [Azure Database for PostgreSQL](https://azure.microsoft.com/services/postgresql/) portal. Azure will manage your backups, high availability through auto-failover, software updates, monitoring, and more for all of your servers. To get started with Hyperscale (Citus), use the [Hyperscale (Citus) Quickstart](https://docs.microsoft.com/azure/postgresql/quickstart-create-hyperscale-portal) in the Azure docs.
 ### Running Citus using Docker
@ -95,14 +93,14 @@ Install packages on Ubuntu / Debian:
 ```bash
 curl https://install.citusdata.com/community/deb.sh > add-citus-repo.sh
 sudo bash add-citus-repo.sh
-sudo apt-get -y install postgresql-17-citus-13.0
+sudo apt-get -y install postgresql-14-citus-11.0
 ```
-Install packages on Red Hat:
+Install packages on CentOS / Fedora / Red Hat:
 ```bash
 curl https://install.citusdata.com/community/rpm.sh > add-citus-repo.sh
 sudo bash add-citus-repo.sh
-sudo yum install -y citus130_17
+sudo yum install -y citus110_14
 ```
 To add Citus to your local PostgreSQL database, add the following to `postgresql.conf`:
@ -236,41 +234,6 @@ Time: 209.961 ms
 Co-location also helps you scale [INSERT..SELECT](https://docs.citusdata.com/en/stable/articles/aggregation.html), [stored procedures](https://www.citusdata.com/blog/2020/11/21/making-postgres-stored-procedures-9x-faster-in-citus/), and [distributed transactions](https://www.citusdata.com/blog/2017/06/02/scaling-complex-sql-transactions/).
 ### Distributing Tables without interrupting the application
 Some of you already start with Postgres, and decide to distribute tables later on while your application using the tables. In that case, you want to avoid downtime for both reads and writes. `create_distributed_table` command block writes (e.g., DML commands) on the table until the command is finished. Instead, with `create_distributed_table_concurrently` command, your application can continue to read and write the data even during the command.
 ```sql
 CREATE TABLE device_logs (
  device_id bigint primary key,
  log text
 );
 -- insert device logs
 INSERT INTO device_logs (device_id, log)
 SELECT s, 'device log:'||s FROM generate_series(0, 99) s;
 -- convert device_logs into a distributed table without interrupting the application
 SELECT create_distributed_table_concurrently('device_logs', 'device_id', colocate_with := 'devices');
 -- get the count of the logs, parallelized across shards
 SELECT count(*) FROM device_logs;
 ┌───────┐
 │ count │
 ├───────┤
 │   100 │
 └───────┘
 (1 row)
 Time: 48.734 ms
 ```
 ### Creating Reference Tables
 When you need fast joins or foreign keys that do not include the distribution column, you can use `create_reference_table` to replicate a table across all nodes in the cluster.
@ -348,72 +311,9 @@ When using columnar storage, you should only load data in batch using `COPY` or
 To learn more about columnar storage, check out the [columnar storage README](https://github.com/citusdata/citus/blob/master/src/backend/columnar/README.md).
 ## Schema-based sharding
 Available since Citus 12.0, [schema-based sharding](https://docs.citusdata.com/en/stable/get_started/concepts.html#schema-based-sharding) is the shared database, separate schema model, the schema becomes the logical shard within the database. Multi-tenant apps can a use a schema per tenant to easily shard along the tenant dimension. Query changes are not required and the application usually only needs a small modification to set the proper search_path when switching tenants. Schema-based sharding is an ideal solution for microservices, and for ISVs deploying applications that cannot undergo the changes required to onboard row-based sharding.
 ### Creating distributed schemas
 You can turn an existing schema into a distributed schema by calling `citus_schema_distribute`:
 ```sql
 SELECT citus_schema_distribute('user_service');
 ```
 Alternatively, you can set `citus.enable_schema_based_sharding` to have all newly created schemas be automatically converted into distributed schemas:
 ```sql
 SET citus.enable_schema_based_sharding TO ON;
 CREATE SCHEMA AUTHORIZATION user_service;
 CREATE SCHEMA AUTHORIZATION time_service;
 CREATE SCHEMA AUTHORIZATION ping_service;
 ```
 ### Running queries
 Queries will be properly routed to schemas based on `search_path` or by explicitly using the schema name in the query.
 For [microservices](https://docs.citusdata.com/en/stable/get_started/tutorial_microservices.html) you would create a USER per service matching the schema name, hence the default `search_path` would contain the schema name. When connected the user queries would be automatically routed and no changes to the microservice would be required.
 ```sql
 CREATE USER user_service;
 CREATE SCHEMA AUTHORIZATION user_service;
 ```
 For typical multi-tenant applications, you would set the search path to the tenant schema name in your application:
 ```sql
 SET search_path = tenant_name, public;
 ```
 ## Setting up with High Availability
 One of the most popular high availability solutions for PostgreSQL, [Patroni 3.0](https://github.com/zalando/patroni), has [first class support for Citus 10.0 and above](https://patroni.readthedocs.io/en/latest/citus.html#citus), additionally since Citus 11.2 ships with improvements for smoother node switchover in Patroni.
 An example of patronictl list output for the Citus cluster:
 ```bash
 postgres@coord1:~$ patronictl list demo
 ```
 ```text
 + Citus cluster: demo ----------+--------------+---------+----+-----------+
 | Group | Member  | Host        | Role         | State   | TL | Lag in MB |
 +-------+---------+-------------+--------------+---------+----+-----------+
 |     0 | coord1  | 172.27.0.10 | Replica      | running |  1 |         0 |
 |     0 | coord2  | 172.27.0.6  | Sync Standby | running |  1 |         0 |
 |     0 | coord3  | 172.27.0.4  | Leader       | running |  1 |           |
 |     1 | work1-1 | 172.27.0.8  | Sync Standby | running |  1 |         0 |
 |     1 | work1-2 | 172.27.0.2  | Leader       | running |  1 |           |
 |     2 | work2-1 | 172.27.0.5  | Sync Standby | running |  1 |         0 |
 |     2 | work2-2 | 172.27.0.7  | Leader       | running |  1 |           |
 +-------+---------+-------------+--------------+---------+----+-----------+
 ```
 ## Documentation
-If you’re ready to get started with Citus or want to know more, we recommend reading the [Citus open source documentation](https://docs.citusdata.com/en/stable/). Or, if you are using Citus on Azure, then the [Azure Cosmos DB for PostgreSQL](https://learn.microsoft.com/azure/cosmos-db/postgresql/introduction) is the place to start.
+If you’re ready to get started with Citus or want to know more, we recommend reading the [Citus open source documentation](https://docs.citusdata.com/en/stable/). Or, if you are using Citus on Azure, then the [Hyperscale (Citus) documentation](https://docs.microsoft.com/azure/postgresql/hyperscale/) is online and available as part of the Azure Database for PostgreSQL docs.
 Our Citus docs contain comprehensive use case guides on how to build a [multi-tenant SaaS application](https://docs.citusdata.com/en/stable/use_cases/multi_tenant.html), [real-time analytics dashboard]( https://docs.citusdata.com/en/stable/use_cases/realtime_analytics.html), or work with [time series data](https://docs.citusdata.com/en/stable/use_cases/timeseries.html).
@ -423,13 +323,11 @@ A Citus database cluster grows from a single PostgreSQL node into a cluster by a
 Data in distributed tables is stored in “shards”, which are actually just regular PostgreSQL tables on the worker nodes. When querying a distributed table on the coordinator node, Citus will send regular SQL queries to the worker nodes. That way, all the usual PostgreSQL optimizations and extensions can automatically be used with Citus.
-![Citus architecture](images/citus-architecture.png)
+![Citus architecture](/citus-architecture.png)
 When you send a query in which all (co-located) distributed tables have the same filter on the distribution column, Citus will automatically detect that and send the whole query to the worker node that stores the data. That way, arbitrarily complex queries are supported with minimal routing overhead, which is especially useful for scaling transactional workloads. If queries do not have a specific filter, each shard is queried in parallel, which is especially useful in analytical workloads. The Citus distributed executor is adaptive and is designed to handle both query types at the same time on the same system under high concurrency, which enables large-scale mixed workloads.
-The schema and metadata of distributed tables and reference tables are automatically synchronized to all the nodes in the cluster. That way, you can connect to any node to run distributed queries. Schema changes and cluster administration still need to go through the coordinator.
+As of Citus 11.0, the schema and metadata of distributed tables and reference tables are automatically synchronized to all the nodes in the cluster. That way, you can connect to any node to run distributed queries. Schema changes and cluster administration still need to go through the coordinator.
 Detailed descriptions of the implementation for Citus developers are provided in the [Citus Technical Documentation](src/backend/distributed/README.md).
 ## When to use Citus
@ -440,23 +338,21 @@ Citus is uniquely capable of scaling both analytical and transactional workloads
  The advanced parallel, distributed query engine in Citus combined with PostgreSQL features such as [array types](https://www.postgresql.org/docs/current/arrays.html), [JSONB](https://www.postgresql.org/docs/current/datatype-json.html), [lateral joins](https://heap.io/blog/engineering/postgresqls-powerful-new-join-type-lateral), and extensions like [HyperLogLog](https://github.com/citusdata/postgresql-hll) and [TopN](https://github.com/citusdata/postgresql-topn) allow you to build responsive analytics dashboards no matter how many customers or how much data you have.
-  Example real-time analytics users: [Algolia](https://www.citusdata.com/customers/algolia)
+  Example real-time analytics users: [Algolia](https://www.citusdata.com/customers/algolia), [Heap](https://www.citusdata.com/customers/heap)
 - **[Time series data](http://docs.citusdata.com/en/stable/use_cases/timeseries.html)**:
  Citus enables you to process and analyze very large amounts of time series data. The biggest Citus clusters store well over a petabyte of time series data and ingest terabytes per day.
  Citus integrates seamlessly with [Postgres table partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html) and has [built-in functions for partitioning by time](https://www.citusdata.com/blog/2021/10/22/how-to-scale-postgres-for-time-series-data-with-citus/), which can speed up queries and writes on time series tables. You can take advantage of Citus’s parallel, distributed query engine for fast analytical queries, and use the built-in *columnar storage* to compress old partitions.
-  Example users: [MixRank](https://www.citusdata.com/customers/mixrank)
+  Example users: [MixRank](https://www.citusdata.com/customers/mixrank), [Windows team](https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/architecting-petabyte-scale-analytics-by-scaling-out-postgres-on/ba-p/969685)
 - **[Software-as-a-service (SaaS) applications](http://docs.citusdata.com/en/stable/use_cases/multi_tenant.html)**:
  SaaS and other multi-tenant applications need to be able to scale their database as the number of tenants/customers grows. Citus enables you to transparently shard a complex data model by the tenant dimension, so your database can grow along with your business.
  By distributing tables along a tenant ID column and co-locating data for the same tenant, Citus can horizontally scale complex (tenant-scoped) queries, transactions, and foreign key graphs. Reference tables and distributed DDL commands make database management a breeze compared to manual sharding. On top of that, you have a built-in distributed query engine for doing cross-tenant analytics inside the database.
-  Example multi-tenant SaaS users: [Salesloft](https://fivetran.com/case-studies/replicating-sharded-databases-a-case-study-of-salesloft-citus-data-and-fivetran), [ConvertFlow](https://www.citusdata.com/customers/convertflow)
+  Example multi-tenant SaaS users: [Copper](https://www.citusdata.com/customers/copper), [Salesloft](https://fivetran.com/case-studies/replicating-sharded-databases-a-case-study-of-salesloft-citus-data-and-fivetran), [ConvertFlow](https://www.citusdata.com/customers/convertflow)
 - **[Microservices](https://docs.citusdata.com/en/stable/get_started/tutorial_microservices.html)**: Citus supports schema based sharding, which allows distributing regular database schemas across many machines. This sharding methodology fits nicely with typical Microservices architecture, where storage is fully owned by the service hence can’t share the same schema definition with other tenants. Citus allows distributing horizontally scalable state across services, solving one of the [main problems](https://stackoverflow.blog/2020/11/23/the-macro-problem-with-microservices/) of microservices.
 - **Geospatial**:
  Because of the powerful [PostGIS](https://postgis.net/) extension to Postgres that adds support for geographic objects into Postgres, many people run spatial/GIS applications on top of Postgres. And since spatial location information has become part of our daily life, well, there are more geospatial applications than ever. When your Postgres database needs to scale out to handle an increased workload, Citus is a good fit.
@ -469,25 +365,19 @@ Citus is uniquely capable of scaling both analytical and transactional workloads
 - **GitHub issues**: Please submit issues via [GitHub issues](https://github.com/citusdata/citus/issues).
 - **Documentation**: Our [Citus docs](https://docs.citusdata.com ) have a wealth of resources, including sections on [query performance tuning](https://docs.citusdata.com/en/stable/performance/performance_tuning.html), [useful diagnostic queries](https://docs.citusdata.com/en/stable/admin_guide/diagnostic_queries.html), and [common error messages](https://docs.citusdata.com/en/stable/reference/common_errors.html).
 - **Docs issues**: You can also submit documentation issues via [GitHub issues for our Citus docs](https://github.com/citusdata/citus_docs/issues).
- **Updates & Release Notes**: Learn about what's new in each Citus version on the [Citus Updates page](https://www.citusdata.com/updates/).
+- **Updates**: Learn about what's new in each Citus version on the [Citus Updates page](https://www.citusdata.com/updates/).
 ## Contributing
 Citus is built on and of open source, and we welcome your contributions. The [CONTRIBUTING.md](CONTRIBUTING.md) file explains how to get started developing the Citus extension itself and our code quality guidelines.
 ## Code of Conduct
 This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
 For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
 contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
 ## Stay Connected
 - **Twitter**: Follow us [@citusdata](https://twitter.com/citusdata) to track the latest posts & updates on what’s happening.
- **Citus Blog**: Read our popular [Citus Open Source Blog](https://www.citusdata.com/blog/) for posts about PostgreSQL and Citus.
+- **Citus Blog**: Read our popular [Citus Blog](https://www.citusdata.com/blog/) for useful & informative posts about PostgreSQL and Citus.
 - **Citus Newsletter**: Subscribe to our monthly technical [Citus Newsletter](https://www.citusdata.com/join-newsletter) to get a curated collection of our favorite posts, videos, docs, talks, & other Postgres goodies.
 - **Slack**: Our [Citus Public slack](https://slack.citusdata.com/) is a good way to stay connected, not just with us but with other Citus users.
- **Sister Blog**: Read the PostgreSQL posts on the [Azure Cosmos DB for PostgreSQL blog](https://devblogs.microsoft.com/cosmosdb/category/postgresql/) about our managed service on Azure.
+- **Sister Blog**: Read our Azure Database for PostgreSQL [sister blog on Microsoft TechCommunity](https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/bg-p/ADforPostgreSQL) for posts relating to Postgres (and Citus) on Azure.
 - **Videos**: Check out this [YouTube playlist](https://www.youtube.com/playlist?list=PLixnExCn6lRq261O0iwo4ClYxHpM9qfVy) of some of our favorite Citus videos and demos. If you want to deep dive into how Citus extends PostgreSQL, you might want to check out Marco Slot’s talk at Carnegie Mellon titled [Citus: Distributed PostgreSQL as an Extension](https://youtu.be/X-aAgXJZRqM) that was part of Andy Pavlo’s Vaccination Database Talks series at CMUDB.
 - **Our other Postgres projects**: Our team also works on other awesome PostgreSQL open source extensions & projects, including: [pg_cron](https://github.com/citusdata/pg_cron), [HyperLogLog](https://github.com/citusdata/postgresql-hll), [TopN](https://github.com/citusdata/postgresql-topn), [pg_auto_failover](https://github.com/citusdata/pg_auto_failover), [activerecord-multi-tenant](https://github.com/citusdata/activerecord-multi-tenant), and [django-multitenant](https://github.com/citusdata/django-multitenant).
--- a/SECURITY.md
+++ b/SECURITY.md
@ -1,41 +0,0 @@
 <!-- BEGIN MICROSOFT SECURITY.MD V0.0.8 BLOCK -->
 ## Security
 Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
 If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
 ## Reporting Security Issues
 **Please do not report security vulnerabilities through public GitHub issues.**
 Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
 If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com).  If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
 You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
 Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
  * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
  * Full paths of source file(s) related to the manifestation of the issue
  * The location of the affected source code (tag/branch/commit or direct URL)
  * Any special configuration required to reproduce the issue
  * Step-by-step instructions to reproduce the issue
  * Proof-of-concept or exploit code (if possible)
  * Impact of the issue, including how an attacker might exploit the issue
 This information will help us triage your report more quickly.
 If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
 ## Preferred Languages
 We prefer all communications to be in English.
 ## Policy
 Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
 <!-- END MICROSOFT SECURITY.MD BLOCK -->
--- a/STYLEGUIDE.md
+++ b/STYLEGUIDE.md
@ -1,160 +0,0 @@
 # Coding style
 The existing code-style in our code-base is not super consistent. There are multiple reasons for that. One big reason is because our code-base is relatively old and our standards have changed over time. The second big reason is that our style-guide is different from style-guide of Postgres and some code is copied from Postgres source code and is slightly modified. The below rules are for new code. If you're changing existing code that uses a different style, use your best judgement to decide if you use the rules here or if you match the existing style.
 ## Using citus_indent
 CI pipeline will automatically reject any PRs which do not follow our coding
 conventions. The easiest way to ensure your PR adheres to those conventions is
 to use the [citus_indent](https://github.com/citusdata/tools/tree/develop/uncrustify)
 tool. This tool uses `uncrustify` under the hood.
 ```bash
 # Uncrustify changes the way it formats code every release a bit. To make sure
 # everyone formats consistently we use version 0.68.1:
 curl -L https://github.com/uncrustify/uncrustify/archive/uncrustify-0.68.1.tar.gz | tar xz
 cd uncrustify-uncrustify-0.68.1/
 mkdir build
 cd build
 cmake ..
 make -j5
 sudo make install
 cd ../..
 git clone https://github.com/citusdata/tools.git
 cd tools
 make uncrustify/.install
 ```
 Once you've done that, you can run the `make reindent` command from the top
 directory to recursively check and correct the style of any source files in the
 current directory. Under the hood, `make reindent` will run `citus_indent` and
 some other style corrections for you.
 You can also run the following in the directory of this repository to
 automatically format all the files that you have changed before committing:
 ```bash
 cat > .git/hooks/pre-commit << __EOF__
 #!/bin/bash
 citus_indent --check --diff || { citus_indent --diff; exit 1; }
 __EOF__
 chmod +x .git/hooks/pre-commit
 ```
 ## Other rules we follow that citus_indent does not enforce
 * We almost always use **CamelCase**, when naming functions, variables etc., **not snake_case**.
 * We also have the habits of using a **lowerCamelCase** for some variables named from their type or from their function name, as shown in the examples:
  ```c
  bool IsCitusExtensionLoaded = false;
  bool
  IsAlterTableRenameStmt(RenameStmt *renameStmt)
  {
    AlterTableCmd *alterTableCommand = NULL;
    ..
    ..
    bool isAlterTableRenameStmt = false;
    ..
  }
  ```
 * We **start functions with a comment**:
  ```c
  /*
   * MyNiceFunction <something in present simple tense, e.g., processes / returns / checks / takes X as input / does Y> ..
   * <some more nice words> ..
   * <some more nice words> ..
   */
  <static?> <return type>
  MyNiceFunction(..)
  {
    ..
    ..
  }
  ```
 * `#includes` needs to be sorted based on below ordering and then alphabetically and we should not include what we don't need in a file:
  * System includes (eg. #include<...>)
  * Postgres.h (eg. #include "postgres.h")
  * Toplevel imports from postgres, not contained in a directory (eg. #include "miscadmin.h")
  * General postgres includes (eg . #include "nodes/...")
  * Toplevel citus includes, not contained in a directory (eg. #include "citus_verion.h")
  * Columnar includes (eg. #include "columnar/...")
  * Distributed includes (eg. #include "distributed/...")
 * Comments:
  ```c
  /* single line comments start with a lower-case */
  /*
   * We start multi-line comments with a capital letter
   * and keep adding a star to the beginning of each line
   * until we close the comment with a star and a slash.
   */
  ```
 * Order of function implementations and their declarations in a file:
  We define static functions after the functions that call them. For example:
  ```c
  #include<..>
  #include<..>
  ..
  ..
  typedef struct
  {
    ..
    ..
  } MyNiceStruct;
  ..
  ..
  PG_FUNCTION_INFO_V1(my_nice_udf1);
  PG_FUNCTION_INFO_V1(my_nice_udf2);
  ..
  ..
  // ..  somewhere on top of the file …
  static void MyNiceStaticlyDeclaredFunction1(…);
  static void MyNiceStaticlyDeclaredFunction2(…);
  ..
  ..
  void
  MyNiceFunctionExternedViaHeaderFile(..)
  {
    ..
    ..
    MyNiceStaticlyDeclaredFunction1(..);
    ..
    ..
    MyNiceStaticlyDeclaredFunction2(..);
    ..
  }
  ..
  ..
  // we define this first because it's called by MyNiceFunctionExternedViaHeaderFile()
  // before MyNiceStaticlyDeclaredFunction2()
  static void
  MyNiceStaticlyDeclaredFunction1(…)
  {
  }
  ..
  ..
  // then we define this
  static void
  MyNiceStaticlyDeclaredFunction2(…)
  {
  }
  ```
--- a/ci/README.md
+++ b/ci/README.md
@ -283,14 +283,6 @@ actually run in CI. This is most commonly forgotten for newly added CI tests
 that the developer only ran locally. It also checks that all CI scripts have a
 section in this `README.md` file and that they include `ci/ci_helpers.sh`.
 ## `check_migration_files.sh`
 A branch that touches a set of upgrade scripts is also expected to touch
 corresponding downgrade scripts as well. If this script fails, read the output
 and make sure you update the downgrade scripts in the printed list. If you
 really don't need a downgrade to run any SQL. You can write a comment in the
 file explaining why a downgrade step is not necessary.
 ## `disallow_c_comments_in_migrations.sh`
 We do not use C-style comments in migration files as the stripped
@ -381,22 +373,3 @@ This script checks and fixes issues with `.gitignore` rules:
 This script checks the order of the GUCs defined in `shared_library_init.c`.
 To solve this failure, please check `shared_library_init.c` and make sure that the GUC
 definitions are in alphabetical order.
 ## `print_stack_trace.sh`
 This script prints stack traces for failed tests, if they left core files.
 ## `sort_and_group_includes.sh`
 This script checks and fixes issues with include grouping and sorting in C files.
 Includes are grouped in the following groups:
 - System includes (eg. `#include <math>`)
 - Postgres.h include (eg. `#include "postgres.h"`)
 - Toplevel postgres includes (includes not in a directory eg. `#include "miscadmin.h`)
 - Postgres includes in a directory (eg. `#include "catalog/pg_type.h"`)
 - Toplevel citus includes (includes not in a directory eg. `#include "pg_version_constants.h"`)
 - Columnar includes (eg. `#include "columnar/columnar.h"`)
 - Distributed includes (eg. `#include "distributed/maintenanced.h"`)
 Within every group the include lines are sorted alphabetically.
--- a/ci/build-citus.sh
+++ b/ci/build-citus.sh
@ -15,6 +15,9 @@ PG_MAJOR=${PG_MAJOR:?please provide the postgres major version}
 codename=${VERSION#*(}
 codename=${codename%)*}
 # get project from argument
 project="${CIRCLE_PROJECT_REPONAME}"
 # we'll do everything with absolute paths
 basedir="$(pwd)"
@ -25,7 +28,7 @@ build_ext() {
  pg_major="$1"
  builddir="${basedir}/build-${pg_major}"
-  echo "Beginning build for PostgreSQL ${pg_major}..." >&2
+  echo "Beginning build of ${project} for PostgreSQL ${pg_major}..." >&2
  # do everything in a subdirectory to avoid clutter in current directory
  mkdir -p "${builddir}" && cd "${builddir}"
--- a/ci/check_all_ci_scripts_are_run.sh
+++ b/ci/check_all_ci_scripts_are_run.sh
@ -14,8 +14,8 @@ ci_scripts=$(
    grep -v -E '^(ci_helpers.sh|fix_style.sh)$'
 )
 for script in $ci_scripts; do
-    if ! grep "\\bci/$script\\b" -r .github > /dev/null; then
+    if ! grep "\\bci/$script\\b" .circleci/config.yml > /dev/null; then
-        echo "ERROR: CI script with name \"$script\" is not actually used in .github folder"
+        echo "ERROR: CI script with name \"$script\" is not actually used in .circleci/config.yml"
        exit 1
    fi
    if ! grep "^## \`$script\`\$" ci/README.md > /dev/null; then
--- a/ci/check_enterprise_merge.sh
+++ b/ci/check_enterprise_merge.sh
@ -0,0 +1,96 @@
 #!/bin/bash
 # Testing this script locally requires you to set the following environment
 # variables:
 # CIRCLE_BRANCH, GIT_USERNAME and GIT_TOKEN
 # fail if trying to reference a variable that is not set.
 set -u
 # exit immediately if a command fails
 set -e
 # Fail on pipe failures
 set -o pipefail
 PR_BRANCH="${CIRCLE_BRANCH}"
 ENTERPRISE_REMOTE="https://${GIT_USERNAME}:${GIT_TOKEN}@github.com/citusdata/citus-enterprise"
 # shellcheck disable=SC1091
 source ci/ci_helpers.sh
 # List executed commands. This is done so debugging this script is easier when
 # it fails. It's explicitly done after git remote add so username and password
 # are not shown in CI output (even though it's also filtered out by CircleCI)
 set -x
 check_compile () {
    echo "INFO: checking if merged code can be compiled"
    ./configure --without-libcurl
    make -j10
 }
 # Clone current git repo (which should be community) to a temporary working
 # directory and go there
 GIT_DIR_ROOT="$(git rev-parse --show-toplevel)"
 TMP_GIT_DIR="$(mktemp --directory -t citus-merge-check.XXXXXXXXX)"
 git clone "$GIT_DIR_ROOT" "$TMP_GIT_DIR"
 cd "$TMP_GIT_DIR"
 # Fails in CI without this
 git config user.email "citus-bot@microsoft.com"
 git config user.name "citus bot"
 # Disable "set -x" temporarily, because $ENTERPRISE_REMOTE contains passwords
 { set +x ; } 2> /dev/null
 git remote add enterprise "$ENTERPRISE_REMOTE"
 set -x
 git remote set-url --push enterprise no-pushing
 # Fetch enterprise-master
 git fetch enterprise enterprise-master
 git checkout "enterprise/enterprise-master"
 if git merge --no-commit "origin/$PR_BRANCH"; then
    echo "INFO: community PR branch could be merged into enterprise-master"
    # check that we can compile after the merge
    if check_compile; then
        exit 0
    fi
    echo "WARN: Failed to compile after community PR branch was merged into enterprise"
 fi
 # undo partial merge
 git merge --abort
 # If we have a conflict on enterprise merge on the master branch, we have a problem.
 # Provide an error message to indicate that enterprise merge is needed to fix this check.
 if [[ $PR_BRANCH = master ]]; then
    echo "ERROR: Master branch has merge conflicts with enterprise-master."
    echo "Try re-running this CI job after merging your changes into enterprise-master."
    exit 1
 fi
 if ! git fetch enterprise "$PR_BRANCH" ; then
    echo "ERROR: enterprise/$PR_BRANCH was not found and community PR branch could not be merged into enterprise-master"
    exit 1
 fi
 # Show the top commit of the enterprise PR branch to make debugging easier
 git log -n 1 "enterprise/$PR_BRANCH"
 # Check that this branch contains the top commit of the current community PR
 # branch. If it does not it means it's not up to date with the current PR, so
 # the enterprise branch should be updated.
 if ! git merge-base --is-ancestor "origin/$PR_BRANCH" "enterprise/$PR_BRANCH" ; then
    echo "ERROR: enterprise/$PR_BRANCH is not up to date with community PR branch"
    exit 1
 fi
 # Now check if we can merge the enterprise PR into enterprise-master without
 # issues.
 git merge --no-commit "enterprise/$PR_BRANCH"
 # check that we can compile after the merge
 check_compile
--- a/ci/check_gucs_are_alphabetically_sorted.sh
+++ b/ci/check_gucs_are_alphabetically_sorted.sh
@ -4,22 +4,7 @@ set -euo pipefail
 # shellcheck disable=SC1091
 source ci/ci_helpers.sh
-# Find the line that exactly matches "RegisterCitusConfigVariables(void)" in
+# extract citus gucs in the form of "citus.X"
-# shared_library_init.c. grep command returns something like
+grep -o -E "(\.*\"citus.\w+\")," src/backend/distributed/shared_library_init.c > gucs.out
-# "934:RegisterCitusConfigVariables(void)" and we extract the line number
+sort -c gucs.out
 # with cut.
 RegisterCitusConfigVariables_begin_linenumber=$(grep -n "^RegisterCitusConfigVariables(void)$" src/backend/distributed/shared_library_init.c | cut -d: -f1)
 # Consider the lines starting from $RegisterCitusConfigVariables_begin_linenumber,
 # grep the first line that starts with "}" and extract the line number with cut
 # as in the previous step.
 RegisterCitusConfigVariables_length=$(tail -n +$RegisterCitusConfigVariables_begin_linenumber src/backend/distributed/shared_library_init.c | grep -n -m 1 "^}$" | cut -d: -f1)
 # extract the function definition of RegisterCitusConfigVariables into a temp file
 tail -n +$RegisterCitusConfigVariables_begin_linenumber src/backend/distributed/shared_library_init.c | head -n $(($RegisterCitusConfigVariables_length)) > RegisterCitusConfigVariables_func_def.out
 # extract citus gucs in the form of <tab><tab>"citus.X"
 grep -P "^[\t][\t]\"citus\.[a-zA-Z_0-9]+\"" RegisterCitusConfigVariables_func_def.out > gucs.out
 LC_COLLATE=C sort -c gucs.out
 rm gucs.out
 rm RegisterCitusConfigVariables_func_def.out
--- a/ci/check_migration_files.sh
+++ b/ci/check_migration_files.sh
@ -1,33 +0,0 @@
 #! /bin/bash
 set -euo pipefail
 # shellcheck disable=SC1091
 source ci/ci_helpers.sh
 # This file checks for the existence of downgrade scripts for every upgrade script that is changed in the branch.
 # create list of migration files for upgrades
 upgrade_files=$(git diff --name-only origin/main | { grep "src/backend/distributed/sql/citus--.*sql" || exit 0 ; })
 downgrade_files=$(git diff --name-only origin/main | { grep "src/backend/distributed/sql/downgrades/citus--.*sql" || exit 0 ; })
 ret_value=0
 for file in $upgrade_files
 do
    # There should always be 2 matches, and no need to avoid splitting here
    # shellcheck disable=SC2207
    versions=($(grep --only-matching --extended-regexp "[0-9]+\.[0-9]+[-.][0-9]+" <<< "$file"))
    from_version=${versions[0]};
    to_version=${versions[1]};
    downgrade_migration_file="src/backend/distributed/sql/downgrades/citus--$to_version--$from_version.sql"
    # check for the existence of migration scripts
    if [[ $(grep --line-regexp --count "$downgrade_migration_file" <<< "$downgrade_files") == 0 ]]
    then
        echo "$file is updated, but $downgrade_migration_file is not updated in branch"
        ret_value=1
    fi
 done
 exit $ret_value;
--- a/ci/fix_gitignore.sh
+++ b/ci/fix_gitignore.sh
@ -5,8 +5,7 @@ source ci/ci_helpers.sh
 # Remove all the ignored files from git tree, and error out
 # find all ignored files in git tree, and use quotation marks to prevent word splitting on filenames with spaces in them
-# NOTE: Option --cached is needed to avoid a bug in git ls-files command.
+ignored_lines_in_git_tree=$(git ls-files --ignored --exclude-standard | sed 's/.*/"&"/')
 ignored_lines_in_git_tree=$(git ls-files --ignored --cached --exclude-standard | sed 's/.*/"&"/')
 if [[ -n $ignored_lines_in_git_tree ]]
 then
--- a/ci/fix_style.sh
+++ b/ci/fix_style.sh
@ -9,8 +9,6 @@ cidir="${0%/*}"
 cd ${cidir}/..
 citus_indent . --quiet
 black . --quiet
 isort . --quiet
 ci/editorconfig.sh
 ci/remove_useless_declarations.sh
 ci/disallow_c_comments_in_migrations.sh
@ -18,5 +16,3 @@ ci/disallow_hash_comments_in_spec_files.sh
 ci/disallow_long_changelog_entries.sh
 ci/normalize_expected.sh
 ci/fix_gitignore.sh
 ci/print_stack_trace.sh
 ci/sort_and_group_includes.sh
--- a/ci/include_grouping.py
+++ b/ci/include_grouping.py
@ -1,157 +0,0 @@
 #!/usr/bin/env python3
 """
 easy command line to run against all citus-style checked files:
 $ git ls-files \
  | git check-attr --stdin citus-style \
  | grep 'citus-style: set' \
  | awk '{print $1}' \
  | cut -d':' -f1 \
  | xargs -n1 ./ci/include_grouping.py
 """
 import collections
 import os
 import sys
 def main(args):
    if len(args) < 2:
        print("Usage: include_grouping.py <file>")
        return
    file = args[1]
    if not os.path.isfile(file):
        sys.exit(f"File '{file}' does not exist")
    with open(file, "r") as in_file:
        with open(file + ".tmp", "w") as out_file:
            includes = []
            skipped_lines = []
            # This calls print_sorted_includes on a set of consecutive #include lines.
            # This implicitly keeps separation of any #include lines that are contained in
            # an #ifdef, because it will order the #include lines inside and after the
            # #ifdef completely separately.
            for line in in_file:
                # if a line starts with #include we don't want to print it yet, instead we
                # want to collect all consecutive #include lines
                if line.startswith("#include"):
                    includes.append(line)
                    skipped_lines = []
                    continue
                # if we have collected any #include lines, we want to print them sorted
                # before printing the current line. However, if the current line is empty
                # we want to perform a lookahead to see if the next line is an #include.
                # To maintain any separation between #include lines and their subsequent
                # lines we keep track of all lines we have skipped inbetween.
                if len(includes) > 0:
                    if len(line.strip()) == 0:
                        skipped_lines.append(line)
                        continue
                    # we have includes that need to be grouped before printing the current
                    # line.
                    print_sorted_includes(includes, file=out_file)
                    includes = []
                    # print any skipped lines
                    print("".join(skipped_lines), end="", file=out_file)
                    skipped_lines = []
                print(line, end="", file=out_file)
    # move out_file to file
    os.rename(file + ".tmp", file)
 def print_sorted_includes(includes, file=sys.stdout):
    default_group_key = 1
    groups = collections.defaultdict(set)
    # define the groups that we separate correctly. The matchers are tested in the order
    # of their priority field. The first matcher that matches the include is used to
    # assign the include to a group.
    # The groups are printed in the order of their group_key.
    matchers = [
        {
            "name": "system includes",
            "matcher": lambda x: x.startswith("<"),
            "group_key": -2,
            "priority": 0,
        },
        {
            "name": "toplevel postgres includes",
            "matcher": lambda x: "/" not in x,
            "group_key": 0,
            "priority": 9,
        },
        {
            "name": "postgres.h",
            "matcher": lambda x: x.strip() in ['"postgres.h"'],
            "group_key": -1,
            "priority": -1,
        },
        {
            "name": "toplevel citus inlcudes",
            "matcher": lambda x: x.strip()
            in [
                '"citus_version.h"',
                '"pg_version_compat.h"',
                '"pg_version_constants.h"',
            ],
            "group_key": 3,
            "priority": 0,
        },
        {
            "name": "columnar includes",
            "matcher": lambda x: x.startswith('"columnar/'),
            "group_key": 4,
            "priority": 1,
        },
        {
            "name": "distributed includes",
            "matcher": lambda x: x.startswith('"distributed/'),
            "group_key": 5,
            "priority": 1,
        },
    ]
    matchers.sort(key=lambda x: x["priority"])
    # throughout our codebase we have some includes where either postgres or citus
    # includes are wrongfully included with the syntax for system includes. Before we
    # try to match those we will change the <> to "" to make them match our system. This
    # will also rewrite the include to the correct syntax.
    common_system_include_error_prefixes = ["<nodes/", "<distributed/"]
    # assign every include to a group
    for include in includes:
        # extract the group key from the include
        include_content = include.split(" ")[1]
        # fix common system includes which are secretly postgres or citus includes
        for common_prefix in common_system_include_error_prefixes:
            if include_content.startswith(common_prefix):
                include_content = '"' + include_content.strip()[1:-1] + '"'
                include = include.split(" ")[0] + " " + include_content + "\n"
                break
        group_key = default_group_key
        for matcher in matchers:
            if matcher["matcher"](include_content):
                group_key = matcher["group_key"]
                break
        groups[group_key].add(include)
    # iterate over all groups in the natural order of its keys
    for i, group in enumerate(sorted(groups.items())):
        if i > 0:
            print(file=file)
        includes = group[1]
        print("".join(sorted(includes)), end="", file=file)
 if __name__ == "__main__":
    main(sys.argv)
--- a/ci/print_stack_trace.sh
+++ b/ci/print_stack_trace.sh
@ -1,25 +0,0 @@
 #!/bin/bash
 set -euo pipefail
 # shellcheck disable=SC1091
 source ci/ci_helpers.sh
 # find all core files
 core_files=( $(find . -type f -regex .*core.*\d*.*postgres) )
 if [ ${#core_files[@]} -gt 0 ]; then
    # print stack traces for the core files
    for core_file in "${core_files[@]}"
    do
        # set print frame-arguments all: show all scalars + structures in the frame
        # set print pretty on:           show structures in indented mode
        # set print addr off:            do not show pointer address
        # thread apply all bt full:      show stack traces for all threads
        gdb --batch \
            -ex "set print frame-arguments all" \
            -ex "set print pretty on" \
            -ex "set print addr off" \
            -ex "thread apply all bt full" \
            postgres "${core_file}"
    done
 fi
--- a/ci/sort_and_group_includes.sh
+++ b/ci/sort_and_group_includes.sh
@ -1,12 +0,0 @@
 #!/bin/bash
 set -euo pipefail
 # shellcheck disable=SC1091
 source ci/ci_helpers.sh
 git ls-files \
  | git check-attr --stdin citus-style \
  | grep 'citus-style: set' \
  | awk '{print $1}' \
  | cut -d':' -f1 \
  | xargs -n1 ./ci/include_grouping.py
--- a/images/citus-architecture.png
+++ b/images/citus-architecture.png
--- a/images/citus-readme-banner.png
+++ b/images/citus-readme-banner.png
--- a/images/citus-scale-out.png
+++ b/images/citus-scale-out.png
--- a/20
+++ b/20
@ -1,6 +1,6 @@
 #! /bin/sh
 # Guess values for system-dependent variables and create Makefiles.
-# Generated by GNU Autoconf 2.69 for Citus 13.2devel.
+# Generated by GNU Autoconf 2.69 for Citus 11.1.2.
 #
 #
 # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
@ -579,8 +579,8 @@ MAKEFLAGS=
 # Identity of this package.
 PACKAGE_NAME='Citus'
 PACKAGE_TARNAME='citus'
-PACKAGE_VERSION='13.2devel'
+PACKAGE_VERSION='11.1.2'
-PACKAGE_STRING='Citus 13.2devel'
+PACKAGE_STRING='Citus 11.1.2'
 PACKAGE_BUGREPORT=''
 PACKAGE_URL=''
@ -1262,7 +1262,7 @@ if test "$ac_init_help" = "long"; then
  # Omit some internal or obsolete options to make the list less imposing.
  # This message is too long to be a string in the A/UX 3.1 sh.
  cat <<_ACEOF
-\`configure' configures Citus 13.2devel to adapt to many kinds of systems.
+\`configure' configures Citus 11.1.2 to adapt to many kinds of systems.
 Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1324,7 +1324,7 @@ fi
 if test -n "$ac_init_help"; then
  case $ac_init_help in
-     short | recursive ) echo "Configuration of Citus 13.2devel:";;
+     short | recursive ) echo "Configuration of Citus 11.1.2:";;
   esac
  cat <<\_ACEOF
@ -1429,7 +1429,7 @@ fi
 test -n "$ac_init_help" && exit $ac_status
 if $ac_init_version; then
  cat <<\_ACEOF
-Citus configure 13.2devel
+Citus configure 11.1.2
 generated by GNU Autoconf 2.69
 Copyright (C) 2012 Free Software Foundation, Inc.
@ -1912,7 +1912,7 @@ cat >config.log <<_ACEOF
 This file contains any messages produced by compilers while
 running configure, to aid debugging if configure makes a mistake.
-It was created by Citus $as_me 13.2devel, which was
+It was created by Citus $as_me 11.1.2, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
  $ $0 $@
@ -2588,7 +2588,7 @@ fi
 if test "$with_pg_version_check" = no; then
    { $as_echo "$as_me:${as_lineno-$LINENO}: building against PostgreSQL $version_num (skipped compatibility check)" >&5
 $as_echo "$as_me: building against PostgreSQL $version_num (skipped compatibility check)" >&6;}
-elif test "$version_num" != '15' -a  "$version_num" != '16' -a  "$version_num" != '17'; then
+elif test "$version_num" != '13' -a "$version_num" != '14' -a "$version_num" != '15'; then
   as_fn_error $? "Citus is not compatible with the detected PostgreSQL version ${version_num}." "$LINENO" 5
 else
   { $as_echo "$as_me:${as_lineno-$LINENO}: building against PostgreSQL $version_num" >&5
@ -5393,7 +5393,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
 # report actual input values of CONFIG_FILES etc. instead of their
 # values after options handling.
 ac_log="
-This file was extended by Citus $as_me 13.2devel, which was
+This file was extended by Citus $as_me 11.1.2, which was
 generated by GNU Autoconf 2.69.  Invocation command line was
  CONFIG_FILES    = $CONFIG_FILES
@ -5455,7 +5455,7 @@ _ACEOF
 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
 ac_cs_version="\\
-Citus config.status 13.2devel
+Citus config.status 11.1.2
 configured by $0, generated by GNU Autoconf 2.69,
  with options \\"\$ac_cs_config\\"
--- a/configure.ac
+++ b/configure.ac
@ -5,7 +5,7 @@
 # everyone needing autoconf installed, the resulting files are checked
 # into the SCM.
-AC_INIT([Citus], [13.2devel])
+AC_INIT([Citus], [11.1.2])
 AC_COPYRIGHT([Copyright (c) Citus Data, Inc.])
 # we'll need sed and awk for some of the version commands
@ -80,7 +80,7 @@ AC_SUBST(with_pg_version_check)
 if test "$with_pg_version_check" = no; then
    AC_MSG_NOTICE([building against PostgreSQL $version_num (skipped compatibility check)])
-elif test "$version_num" != '15' -a  "$version_num" != '16' -a  "$version_num" != '17'; then
+elif test "$version_num" != '13' -a "$version_num" != '14' -a "$version_num" != '15'; then
   AC_MSG_ERROR([Citus is not compatible with the detected PostgreSQL version ${version_num}.])
 else
   AC_MSG_NOTICE([building against PostgreSQL $version_num])
--- a/images/2pc-recovery.png
+++ b/images/2pc-recovery.png
--- a/images/coordinator_delegates_stored_procedure.png
+++ b/images/coordinator_delegates_stored_procedure.png
--- a/images/deadlock-detection.png
+++ b/images/deadlock-detection.png
--- a/images/executor-connections.png
+++ b/images/executor-connections.png
--- a/images/executor-slow-start.png
+++ b/images/executor-slow-start.png
--- a/images/insert-select-modes.png
+++ b/images/insert-select-modes.png
--- a/images/mx-dedicated-query-nodes.png
+++ b/images/mx-dedicated-query-nodes.png
--- a/images/single-repartition-join.png
+++ b/images/single-repartition-join.png
--- a/pyproject.toml
+++ b/pyproject.toml
@ -1,40 +0,0 @@
 [tool.isort]
 profile = 'black'
 [tool.black]
 include = '(src/test/regress/bin/diff-filter|\.pyi?|\.ipynb)$'
 [tool.pytest.ini_options]
 addopts = [
    "--import-mode=importlib",
    "--showlocals",
    "--tb=short",
 ]
 pythonpath = 'src/test/regress/citus_tests'
 asyncio_mode = 'auto'
 # Make test discovery quicker from the root dir of the repo
 testpaths = ['src/test/regress/citus_tests/test']
 # Make test discovery quicker from other directories than root directory
 norecursedirs = [
    '*.egg',
    '.*',
    'build',
    'venv',
    'ci',
    'vendor',
    'backend',
    'bin',
    'include',
    'tmp_*',
    'results',
    'expected',
    'sql',
    'spec',
    'data',
    '__pycache__',
 ]
 # Don't find files with test at the end such as run_test.py
 python_files = ['test_*.py']
--- a/src/backend/columnar/.gitignore
+++ b/src/backend/columnar/.gitignore
@ -1,3 +0,0 @@
 # The directory used to store columnar sql files after pre-processing them
 # with 'cpp' in build-time, see src/backend/columnar/Makefile.
 /build/
--- a/src/backend/columnar/Makefile
+++ b/src/backend/columnar/Makefile
@ -10,51 +10,14 @@ OBJS += \
 MODULE_big = citus_columnar
 EXTENSION = citus_columnar
-template_sql_files = $(patsubst $(citus_abs_srcdir)/%,%,$(wildcard $(citus_abs_srcdir)/sql/*.sql))
+columnar_sql_files = $(patsubst $(citus_abs_srcdir)/%,%,$(wildcard $(citus_abs_srcdir)/sql/*.sql))
-template_downgrade_sql_files = $(patsubst $(citus_abs_srcdir)/sql/downgrades/%,%,$(wildcard $(citus_abs_srcdir)/sql/downgrades/*.sql))
+columnar_downgrade_sql_files = $(patsubst $(citus_abs_srcdir)/%,%,$(wildcard $(citus_abs_srcdir)/sql/downgrades/*.sql))
-generated_sql_files = $(patsubst %,$(citus_abs_srcdir)/build/%,$(template_sql_files))
+DATA = $(columnar_sql_files) \
-generated_downgrade_sql_files += $(patsubst %,$(citus_abs_srcdir)/build/sql/%,$(template_downgrade_sql_files))
+		$(columnar_downgrade_sql_files)
 DATA_built = $(generated_sql_files)
 PG_CPPFLAGS += -I$(libpq_srcdir) -I$(safestringlib_srcdir)/include
 include $(citus_top_builddir)/Makefile.global
-SQL_DEPDIR=.deps/sql
+.PHONY: install-all
 SQL_BUILDDIR=build/sql
 $(generated_sql_files): $(citus_abs_srcdir)/build/%: %
 	@mkdir -p $(citus_abs_srcdir)/$(SQL_DEPDIR) $(citus_abs_srcdir)/$(SQL_BUILDDIR)
 	@# -MF is used to store dependency files(.Po) in another directory for separation
 	@# -MT is used to change the target of the rule emitted by dependency generation.
 	@# -P is used to inhibit generation of linemarkers in the output from the preprocessor.
 	@# -undef is used to not predefine any system-specific or GCC-specific macros.
 	@# `man cpp` for further information
 	cd $(citus_abs_srcdir) && cpp -undef -w -P -MMD -MP -MF$(SQL_DEPDIR)/$(*F).Po -MT$@ $< > $@
 $(generated_downgrade_sql_files): $(citus_abs_srcdir)/build/sql/%: sql/downgrades/%
 	@mkdir -p $(citus_abs_srcdir)/$(SQL_DEPDIR) $(citus_abs_srcdir)/$(SQL_BUILDDIR)
 	@# -MF is used to store dependency files(.Po) in another directory for separation
 	@# -MT is used to change the target of the rule emitted by dependency generation.
 	@# -P is used to inhibit generation of linemarkers in the output from the preprocessor.
 	@# -undef is used to not predefine any system-specific or GCC-specific macros.
 	@# `man cpp` for further information
 	cd $(citus_abs_srcdir) && cpp -undef -w -P -MMD -MP -MF$(SQL_DEPDIR)/$(*F).Po -MT$@ $< > $@
 .PHONY: install install-downgrades install-all
 cleanup-before-install:
 	rm -f $(DESTDIR)$(datadir)/$(datamoduledir)/citus_columnar.control
 	rm -f $(DESTDIR)$(datadir)/$(datamoduledir)/columnar--*
 	rm -f $(DESTDIR)$(datadir)/$(datamoduledir)/citus_columnar--*
 install: cleanup-before-install
 # install and install-downgrades should be run sequentially
 install-all: install
 	$(MAKE) install-downgrades
 install-downgrades: $(generated_downgrade_sql_files)
 	$(INSTALL_DATA) $(generated_downgrade_sql_files) '$(DESTDIR)$(datadir)/$(datamoduledir)/'
--- a/src/backend/columnar/README.md
+++ b/src/backend/columnar/README.md
@ -52,7 +52,8 @@ Benefits of Citus Columnar over cstore_fdw:
  ... FOR UPDATE``)
 * No support for serializable isolation level
 * Support for PostgreSQL server versions 12+ only
-* No support for foreign keys
+* No support for foreign keys, unique constraints, or exclusion
  constraints
 * No support for logical decoding
 * No support for intra-node parallel scans
 * No support for ``AFTER ... FOR EACH ROW`` triggers
@ -233,14 +234,16 @@ CREATE TABLE perf_columnar(LIKE perf_row) USING COLUMNAR;
 ## Data
 ```sql
-CREATE OR REPLACE FUNCTION random_words(n INT4) RETURNS TEXT LANGUAGE sql AS $$
+CREATE OR REPLACE FUNCTION random_words(n INT4) RETURNS TEXT LANGUAGE plpython2u AS $$
-  WITH words(w) AS (
+import random
-    SELECT ARRAY['zero','one','two','three','four','five','six','seven','eight','nine','ten']
+t = ''
-  ),
+words = ['zero','one','two','three','four','five','six','seven','eight','nine','ten']
-  random (word) AS (
+for i in xrange(0,n):
-    SELECT w[(random()*array_length(w, 1))::int] FROM generate_series(1, $1) AS i, words
+  if (i != 0):
-  )
+    t += ' '
-  SELECT string_agg(word, ' ') FROM random;
+  r = random.randint(0,len(words)-1)
  t += words[r]
 return t
 $$;
 ```
--- a/src/backend/columnar/citus_columnar.control
+++ b/src/backend/columnar/citus_columnar.control
@ -1,6 +1,6 @@
 # Columnar extension
 comment = 'Citus Columnar extension'
-default_version = '12.2-1'
+default_version = '11.1-1'
 module_pathname = '$libdir/citus_columnar'
 relocatable = false
 schema = pg_catalog
--- a/src/backend/columnar/columnar.c
+++ b/src/backend/columnar/columnar.c
@ -11,18 +11,16 @@
 *-------------------------------------------------------------------------
 */
 #include "postgres.h"
 #include <sys/stat.h>
 #include <unistd.h>
 #include "postgres.h"
 #include "miscadmin.h"
 #include "utils/guc.h"
 #include "utils/rel.h"
 #include "citus_version.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_tableam.h"
--- a/src/backend/columnar/columnar_compression.c
+++ b/src/backend/columnar/columnar_compression.c
@ -13,22 +13,16 @@
 */
 #include "postgres.h"
 #include "citus_version.h"
 #include "common/pg_lzcompress.h"
 #include "lib/stringinfo.h"
 #include "citus_version.h"
 #include "pg_version_constants.h"
 #include "columnar/columnar_compression.h"
 #if HAVE_CITUS_LIBLZ4
 #include <lz4.h>
 #endif
 #if PG_VERSION_NUM >= PG_VERSION_16
 #include "varatt.h"
 #endif
 #if HAVE_LIBZSTD
 #include <zstd.h>
 #endif
--- a/src/backend/columnar/columnar_customscan.c
+++ b/src/backend/columnar/columnar_customscan.c
@ -10,17 +10,18 @@
 *-------------------------------------------------------------------------
 */
-#include <math.h>
+#include "citus_version.h"
 #include "postgres.h"
-#include "miscadmin.h"
+#include <math.h>
 #include "access/amapi.h"
 #include "access/skey.h"
 #include "catalog/pg_am.h"
 #include "catalog/pg_statistic.h"
 #include "commands/defrem.h"
 #include "miscadmin.h"
 #include "nodes/extensible.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
@ -32,10 +33,6 @@
 #include "optimizer/paths.h"
 #include "optimizer/plancat.h"
 #include "optimizer/restrictinfo.h"
 #if PG_VERSION_NUM >= PG_VERSION_16
 #include "parser/parse_relation.h"
 #include "parser/parsetree.h"
 #endif
 #include "utils/builtins.h"
 #include "utils/lsyscache.h"
 #include "utils/relcache.h"
@ -43,13 +40,10 @@
 #include "utils/selfuncs.h"
 #include "utils/spccache.h"
 #include "citus_version.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_customscan.h"
 #include "columnar/columnar_metadata.h"
 #include "columnar/columnar_tableam.h"
 #include "distributed/listutils.h"
 /*
@ -133,9 +127,6 @@ static List * set_deparse_context_planstate(List *dpcontext, Node *node,
 /* other helpers */
 static List * ColumnarVarNeeded(ColumnarScanState *columnarScanState);
 static Bitmapset * ColumnarAttrNeeded(ScanState *ss);
 #if PG_VERSION_NUM >= PG_VERSION_16
 static Bitmapset * fixup_inherited_columns(Oid parentId, Oid childId, Bitmapset *columns);
 #endif
 /* saved hook value in case of unload */
 static set_rel_pathlist_hook_type PreviousSetRelPathlistHook = NULL;
@ -207,7 +198,7 @@ columnar_customscan_init()
 		&EnableColumnarCustomScan,
 		true,
 		PGC_USERSET,
-		GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE,
+		GUC_NO_SHOW_ALL,
 		NULL, NULL, NULL);
 	DefineCustomBoolVariable(
 		"columnar.enable_qual_pushdown",
@ -217,7 +208,7 @@ columnar_customscan_init()
 		&EnableColumnarQualPushdown,
 		true,
 		PGC_USERSET,
-		GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE,
+		GUC_NO_SHOW_ALL,
 		NULL, NULL, NULL);
 	DefineCustomRealVariable(
 		"columnar.qual_pushdown_correlation_threshold",
@ -231,7 +222,7 @@ columnar_customscan_init()
 		0.0,
 		1.0,
 		PGC_USERSET,
-		GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE,
+		GUC_NO_SHOW_ALL,
 		NULL, NULL, NULL);
 	DefineCustomIntVariable(
 		"columnar.max_custom_scan_paths",
@ -243,7 +234,7 @@ columnar_customscan_init()
 		1,
 		1024,
 		PGC_USERSET,
-		GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE,
+		GUC_NO_SHOW_ALL,
 		NULL, NULL, NULL);
 	DefineCustomEnumVariable(
 		"columnar.planner_debug_level",
@ -363,7 +354,7 @@ ColumnarGetRelationInfoHook(PlannerInfo *root, Oid relationObjectId,
 		/* disable index-only scan */
 		IndexOptInfo *indexOptInfo = NULL;
-		foreach_declared_ptr(indexOptInfo, rel->indexlist)
+		foreach_ptr(indexOptInfo, rel->indexlist)
 		{
 			memset(indexOptInfo->canreturn, false, indexOptInfo->ncolumns * sizeof(bool));
 		}
@ -381,7 +372,7 @@ RemovePathsByPredicate(RelOptInfo *rel, PathPredicate removePathPredicate)
 	List *filteredPathList = NIL;
 	Path *path = NULL;
-	foreach_declared_ptr(path, rel->pathlist)
+	foreach_ptr(path, rel->pathlist)
 	{
 		if (!removePathPredicate(path))
 		{
@ -428,7 +419,7 @@ static void
 CostColumnarPaths(PlannerInfo *root, RelOptInfo *rel, Oid relationId)
 {
 	Path *path = NULL;
-	foreach_declared_ptr(path, rel->pathlist)
+	foreach_ptr(path, rel->pathlist)
 	{
 		if (IsA(path, IndexPath))
 		{
@ -544,7 +535,7 @@ ColumnarIndexScanAdditionalCost(PlannerInfo *root, RelOptInfo *rel,
 	 * "anti-correlated" (-1) since both help us avoiding from reading the
 	 * same stripe again and again.
 	 */
-	double absIndexCorrelation = float_abs(indexCorrelation);
+	double absIndexCorrelation = Abs(indexCorrelation);
 	/*
 	 * To estimate the number of stripes that we need to read, we do linear
@ -663,7 +654,7 @@ CheckVarStats(PlannerInfo *root, Var *var, Oid sortop, float4 *absVarCorrelation
 	 * If the Var is not highly correlated, then the chunk's min/max bounds
 	 * will be nearly useless.
 	 */
-	if (float_abs(varCorrelation) < ColumnarQualPushdownCorrelationThreshold)
+	if (Abs(varCorrelation) < ColumnarQualPushdownCorrelationThreshold)
 	{
 		if (absVarCorrelation)
 		{
@ -671,7 +662,7 @@ CheckVarStats(PlannerInfo *root, Var *var, Oid sortop, float4 *absVarCorrelation
 			 * Report absVarCorrelation if caller wants to know why given
 			 * var is rejected.
 			 */
-			*absVarCorrelation = float_abs(varCorrelation);
+			*absVarCorrelation = Abs(varCorrelation);
 		}
 		return false;
 	}
@ -783,7 +774,7 @@ ExtractPushdownClause(PlannerInfo *root, RelOptInfo *rel, Node *node)
 		List *pushdownableArgs = NIL;
 		Node *boolExprArg = NULL;
-		foreach_declared_ptr(boolExprArg, boolExpr->args)
+		foreach_ptr(boolExprArg, boolExpr->args)
 		{
 			Expr *pushdownableArg = ExtractPushdownClause(root, rel,
 														  (Node *) boolExprArg);
@ -1051,15 +1042,6 @@ FindCandidateRelids(PlannerInfo *root, RelOptInfo *rel, List *joinClauses)
 	candidateRelids = bms_del_members(candidateRelids, rel->relids);
 	candidateRelids = bms_del_members(candidateRelids, rel->lateral_relids);
 	/*
 	 * For the relevant PG16 commit requiring this addition:
 	 * postgres/postgres@2489d76
 	 */
 #if PG_VERSION_NUM >= PG_VERSION_16
 	candidateRelids = bms_del_members(candidateRelids, root->outer_join_rels);
 #endif
 	return candidateRelids;
 }
@ -1321,8 +1303,11 @@ AddColumnarScanPath(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
 	cpath->methods = &ColumnarScanPathMethods;
 #if (PG_VERSION_NUM >= PG_VERSION_15)
 	/* necessary to avoid extra Result node in PG15 */
 	cpath->flags = CUSTOMPATH_SUPPORT_PROJECTION;
 #endif
 	/*
 	 * populate generic path information
@ -1386,43 +1371,7 @@ AddColumnarScanPath(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
 		cpath->custom_private = list_make2(NIL, NIL);
 	}
-	int numberOfColumnsRead = 0;
+	int numberOfColumnsRead = bms_num_members(rte->selectedCols);
 #if PG_VERSION_NUM >= PG_VERSION_16
 	if (rte->perminfoindex > 0)
 	{
 		/*
 		 * If perminfoindex > 0, that means that this relation's permission info
 		 * is directly found in the list of rteperminfos of the Query(root->parse)
 		 * So, all we have to do here is retrieve that info.
 		 */
 		RTEPermissionInfo *perminfo = getRTEPermissionInfo(root->parse->rteperminfos,
 														   rte);
 		numberOfColumnsRead = bms_num_members(perminfo->selectedCols);
 	}
 	else
 	{
 		/*
 		 * If perminfoindex = 0, that means we are skipping the check for permission info
 		 * for this relation, which means that it's either a partition or an inheritance child.
 		 * In these cases, we need to access the permission info of the top parent of this relation.
 		 * After thorough checking, we found that the index of the top parent pointing to the correct
 		 * range table entry in Query's range tables (root->parse->rtable) is found under
 		 * RelOptInfo rel->top_parent->relid.
 		 * For reference, check expand_partitioned_rtentry and expand_inherited_rtentry PG functions
 		 */
 		Assert(rel->top_parent);
 		RangeTblEntry *parent_rte = rt_fetch(rel->top_parent->relid, root->parse->rtable);
 		RTEPermissionInfo *perminfo = getRTEPermissionInfo(root->parse->rteperminfos,
 														   parent_rte);
 		numberOfColumnsRead = bms_num_members(fixup_inherited_columns(perminfo->relid,
 																	  rte->relid,
 																	  perminfo->
 																	  selectedCols));
 	}
 #else
 	numberOfColumnsRead = bms_num_members(rte->selectedCols);
 #endif
 	int numberOfClausesPushed = list_length(allClauses);
 	CostColumnarScan(root, rel, rte->relid, cpath, numberOfColumnsRead,
@ -1442,69 +1391,6 @@ AddColumnarScanPath(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
 }
 #if PG_VERSION_NUM >= PG_VERSION_16
 /*
 * fixup_inherited_columns
 *
 * Exact function Copied from PG16 as it's static.
 *
 * When user is querying on a table with children, it implicitly accesses
 * child tables also. So, we also need to check security label of child
 * tables and columns, but there is no guarantee attribute numbers are
 * same between the parent and children.
 * It returns a bitmapset which contains attribute number of the child
 * table based on the given bitmapset of the parent.
 */
 static Bitmapset *
 fixup_inherited_columns(Oid parentId, Oid childId, Bitmapset *columns)
 {
 	Bitmapset *result = NULL;
 	/*
 	 * obviously, no need to do anything here
 	 */
 	if (parentId == childId)
 	{
 		return columns;
 	}
 	int index = -1;
 	while ((index = bms_next_member(columns, index)) >= 0)
 	{
 		/* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
 		AttrNumber attno = index + FirstLowInvalidHeapAttributeNumber;
 		/*
 		 * whole-row-reference shall be fixed-up later
 		 */
 		if (attno == InvalidAttrNumber)
 		{
 			result = bms_add_member(result, index);
 			continue;
 		}
 		char *attname = get_attname(parentId, attno, false);
 		attno = get_attnum(childId, attname);
 		if (attno == InvalidAttrNumber)
 		{
 			elog(ERROR, "cache lookup failed for attribute %s of relation %u",
 				 attname, childId);
 		}
 		result = bms_add_member(result,
 								attno - FirstLowInvalidHeapAttributeNumber);
 		pfree(attname);
 	}
 	return result;
 }
 #endif
 /*
 * CostColumnarScan calculates the cost of scanning the columnar table. The
 * cost is estimated by using all stripe metadata to estimate based on the
@ -1549,14 +1435,13 @@ ColumnarPerStripeScanCost(RelOptInfo *rel, Oid relationId, int numberOfColumnsRe
 		ereport(ERROR, (errmsg("could not open relation with OID %u", relationId)));
 	}
-	List *stripeList = StripesForRelfilelocator(RelationPhysicalIdentifier_compat(
+	List *stripeList = StripesForRelfilenode(relation->rd_node);
 													relation));
 	RelationClose(relation);
 	uint32 maxColumnCount = 0;
 	uint64 totalStripeSize = 0;
 	StripeMetadata *stripeMetadata = NULL;
-	foreach_declared_ptr(stripeMetadata, stripeList)
+	foreach_ptr(stripeMetadata, stripeList)
 	{
 		totalStripeSize += stripeMetadata->dataLength;
 		maxColumnCount = Max(maxColumnCount, stripeMetadata->columnCount);
@ -1607,8 +1492,7 @@ ColumnarTableStripeCount(Oid relationId)
 		ereport(ERROR, (errmsg("could not open relation with OID %u", relationId)));
 	}
-	List *stripeList = StripesForRelfilelocator(RelationPhysicalIdentifier_compat(
+	List *stripeList = StripesForRelfilenode(relation->rd_node);
 													relation));
 	int stripeCount = list_length(stripeList);
 	RelationClose(relation);
@ -1930,6 +1814,11 @@ ColumnarScan_EndCustomScan(CustomScanState *node)
 	 */
 	TableScanDesc scanDesc = node->ss.ss_currentScanDesc;
 	/*
 	 * Free the exprcontext
 	 */
 	ExecFreeExprContext(&node->ss.ps);
 	/*
 	 * clean out the tuple table
 	 */
--- a/src/backend/columnar/columnar_debug.c
+++ b/src/backend/columnar/columnar_debug.c
@ -11,12 +11,12 @@
 #include "postgres.h"
 #include "funcapi.h"
 #include "miscadmin.h"
 #include "access/nbtree.h"
 #include "access/table.h"
 #include "catalog/pg_am.h"
 #include "catalog/pg_type.h"
 #include "distributed/pg_version_constants.h"
 #include "miscadmin.h"
 #include "storage/fd.h"
 #include "storage/smgr.h"
 #include "utils/guc.h"
@ -25,8 +25,6 @@
 #include "utils/tuplestore.h"
 #include "pg_version_compat.h"
 #include "pg_version_constants.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_storage.h"
 #include "columnar/columnar_version_compat.h"
@ -161,5 +159,5 @@ MemoryContextTotals(MemoryContext context, MemoryContextCounters *counters)
 		MemoryContextTotals(child, counters);
 	}
-	context->methods->stats(context, NULL, NULL, counters, true);
+	context->methods->stats_compat(context, NULL, NULL, counters, true);
 }
--- a/src/backend/columnar/columnar_metadata.c
+++ b/src/backend/columnar/columnar_metadata.c
@ -19,62 +19,47 @@
 */
 #include <sys/stat.h>
 #include "postgres.h"
 #include "miscadmin.h"
 #include "port.h"
 #include "safe_lib.h"
 #include "citus_version.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_storage.h"
 #include "columnar/columnar_version_compat.h"
 #include "distributed/listutils.h"
 #include <sys/stat.h>
 #include "access/heapam.h"
 #include "access/htup_details.h"
 #include "access/nbtree.h"
 #include "access/xact.h"
 #include "catalog/indexing.h"
 #include "catalog/namespace.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_namespace.h"
 #include "catalog/pg_collation.h"
 #include "catalog/pg_type.h"
 #include "catalog/namespace.h"
 #include "commands/defrem.h"
 #include "commands/sequence.h"
 #include "commands/trigger.h"
 #include "executor/executor.h"
 #include "executor/spi.h"
-#include "lib/stringinfo.h"
+#include "miscadmin.h"
 #include "nodes/execnodes.h"
 #include "lib/stringinfo.h"
 #include "port.h"
 #include "storage/fd.h"
 #include "storage/lmgr.h"
 #include "storage/procarray.h"
 #include "storage/smgr.h"
 #include "utils/builtins.h"
 #include "utils/fmgroids.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/lsyscache.h"
 #include "utils/rel.h"
 #include "citus_version.h"
 #include "pg_version_constants.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_storage.h"
 #include "columnar/columnar_version_compat.h"
 #include "distributed/listutils.h"
 #if PG_VERSION_NUM >= PG_VERSION_16
 #include "parser/parse_relation.h"
 #include "storage/relfilelocator.h"
 #include "utils/relfilenumbermap.h"
 #else
 #include "utils/relfilenodemap.h"
 #endif
 #define COLUMNAR_RELOPTION_NAMESPACE "columnar"
 #define SLOW_METADATA_ACCESS_WARNING \
 	"Metadata index %s is not available, this might mean slower read/writes " \
 	"on columnar tables. This is expected during Postgres upgrades and not " \
 	"expected otherwise."
 typedef struct
 {
@ -123,7 +108,7 @@ static Oid ColumnarChunkGroupRelationId(void);
 static Oid ColumnarChunkIndexRelationId(void);
 static Oid ColumnarChunkGroupIndexRelationId(void);
 static Oid ColumnarNamespaceId(void);
-static uint64 LookupStorageId(RelFileLocator relfilelocator);
+static uint64 LookupStorageId(RelFileNode relfilenode);
 static uint64 GetHighestUsedRowNumber(uint64 storageId);
 static void DeleteStorageFromColumnarMetadataTable(Oid metadataTableId,
 												   AttrNumber storageIdAtrrNumber,
@ -602,15 +587,14 @@ ReadColumnarOptions(Oid regclass, ColumnarOptions *options)
 * of columnar.chunk.
 */
 void
-SaveStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
+SaveStripeSkipList(RelFileNode relfilenode, uint64 stripe, StripeSkipList *chunkList,
 				   StripeSkipList *chunkList,
 				   TupleDesc tupleDescriptor)
 {
 	uint32 columnIndex = 0;
 	uint32 chunkIndex = 0;
 	uint32 columnCount = chunkList->columnCount;
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	Oid columnarChunkOid = ColumnarChunkRelationId();
 	Relation columnarChunk = table_open(columnarChunkOid, RowExclusiveLock);
 	ModifyState *modifyState = StartModifyRelation(columnarChunk);
@ -669,10 +653,10 @@ SaveStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
 * SaveChunkGroups saves the metadata for given chunk groups in columnar.chunk_group.
 */
 void
-SaveChunkGroups(RelFileLocator relfilelocator, uint64 stripe,
+SaveChunkGroups(RelFileNode relfilenode, uint64 stripe,
 				List *chunkGroupRowCounts)
 {
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	Oid columnarChunkGroupOid = ColumnarChunkGroupRelationId();
 	Relation columnarChunkGroup = table_open(columnarChunkGroupOid, RowExclusiveLock);
 	ModifyState *modifyState = StartModifyRelation(columnarChunkGroup);
@ -705,8 +689,7 @@ SaveChunkGroups(RelFileLocator relfilelocator, uint64 stripe,
 * ReadStripeSkipList fetches chunk metadata for a given stripe.
 */
 StripeSkipList *
-ReadStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
+ReadStripeSkipList(RelFileNode relfilenode, uint64 stripe, TupleDesc tupleDescriptor,
 				   TupleDesc tupleDescriptor,
 				   uint32 chunkCount, Snapshot snapshot)
 {
 	int32 columnIndex = 0;
@ -714,27 +697,19 @@ ReadStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
 	uint32 columnCount = tupleDescriptor->natts;
 	ScanKeyData scanKey[2];
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	Oid columnarChunkOid = ColumnarChunkRelationId();
 	Relation columnarChunk = table_open(columnarChunkOid, AccessShareLock);
 	Relation index = index_open(ColumnarChunkIndexRelationId(), AccessShareLock);
 	ScanKeyInit(&scanKey[0], Anum_columnar_chunk_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, UInt64GetDatum(storageId));
 	ScanKeyInit(&scanKey[1], Anum_columnar_chunk_stripe,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(stripe));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(stripe));
-	Oid indexId = ColumnarChunkIndexRelationId();
+	SysScanDesc scanDescriptor = systable_beginscan_ordered(columnarChunk, index,
-	bool indexOk = OidIsValid(indexId);
+															snapshot, 2, scanKey);
 	SysScanDesc scanDescriptor = systable_beginscan(columnarChunk, indexId,
 													indexOk, snapshot, 2, scanKey);
 	static bool loggedSlowMetadataAccessWarning = false;
 	if (!indexOk && !loggedSlowMetadataAccessWarning)
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING, "chunk_pkey")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	StripeSkipList *chunkList = palloc0(sizeof(StripeSkipList));
 	chunkList->chunkCount = chunkCount;
@ -746,7 +721,8 @@ ReadStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
 			palloc0(chunkCount * sizeof(ColumnChunkSkipNode));
 	}
-	while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
+	while (HeapTupleIsValid(heapTuple = systable_getnext_ordered(scanDescriptor,
 																 ForwardScanDirection)))
 	{
 		Datum datumArray[Natts_columnar_chunk];
 		bool isNullArray[Natts_columnar_chunk];
@ -811,7 +787,8 @@ ReadStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
 		}
 	}
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	index_close(index, AccessShareLock);
 	table_close(columnarChunk, AccessShareLock);
 	chunkList->chunkGroupRowCounts =
@ -822,9 +799,9 @@ ReadStripeSkipList(RelFileLocator relfilelocator, uint64 stripe,
 /*
- * FindStripeByRowNumber returns StripeMetadata for the stripe that has the
+ * FindStripeByRowNumber returns StripeMetadata for the stripe whose
- * smallest firstRowNumber among the stripes whose firstRowNumber is grater
+ * firstRowNumber is greater than given rowNumber. If no such stripe
- * than given rowNumber. If no such stripe exists, then returns NULL.
+ * exists, then returns NULL.
 */
 StripeMetadata *
 FindNextStripeByRowNumber(Relation relation, uint64 rowNumber, Snapshot snapshot)
@ -914,7 +891,8 @@ StripeGetHighestRowNumber(StripeMetadata *stripeMetadata)
 /*
 * StripeMetadataLookupRowNumber returns StripeMetadata for the stripe whose
 * firstRowNumber is less than or equal to (FIND_LESS_OR_EQUAL), or is
- * greater than (FIND_GREATER) given rowNumber.
+ * greater than (FIND_GREATER) given rowNumber by doing backward index
 * scan on stripe_first_row_number_idx.
 * If no such stripe exists, then returns NULL.
 */
 static StripeMetadata *
@ -928,7 +906,7 @@ StripeMetadataLookupRowNumber(Relation relation, uint64 rowNumber, Snapshot snap
 	uint64 storageId = ColumnarStorageGetStorageId(relation, false);
 	ScanKeyData scanKey[2];
 	ScanKeyInit(&scanKey[0], Anum_columnar_stripe_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(storageId));
 	StrategyNumber strategyNumber = InvalidStrategy;
 	RegProcedure procedure = InvalidOid;
@ -943,25 +921,16 @@ StripeMetadataLookupRowNumber(Relation relation, uint64 rowNumber, Snapshot snap
 		procedure = F_INT8GT;
 	}
 	ScanKeyInit(&scanKey[1], Anum_columnar_stripe_first_row_number,
-				strategyNumber, procedure, Int64GetDatum(rowNumber));
+				strategyNumber, procedure, UInt64GetDatum(rowNumber));
 	Relation columnarStripes = table_open(ColumnarStripeRelationId(), AccessShareLock);
 	Relation index = index_open(ColumnarStripeFirstRowNumberIndexRelationId(),
 								AccessShareLock);
 	SysScanDesc scanDescriptor = systable_beginscan_ordered(columnarStripes, index,
 															snapshot, 2,
 															scanKey);
 	Oid indexId = ColumnarStripeFirstRowNumberIndexRelationId();
 	bool indexOk = OidIsValid(indexId);
 	SysScanDesc scanDescriptor = systable_beginscan(columnarStripes, indexId, indexOk,
 													snapshot, 2, scanKey);
 	static bool loggedSlowMetadataAccessWarning = false;
 	if (!indexOk && !loggedSlowMetadataAccessWarning)
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING,
 								 "stripe_first_row_number_idx")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	if (indexOk)
 	{
 	ScanDirection scanDirection = NoMovementScanDirection;
 	if (lookupMode == FIND_LESS_OR_EQUAL)
 	{
@ -976,40 +945,9 @@ StripeMetadataLookupRowNumber(Relation relation, uint64 rowNumber, Snapshot snap
 	{
 		foundStripeMetadata = BuildStripeMetadata(columnarStripes, heapTuple);
 	}
 	}
 	else
 	{
 		HeapTuple heapTuple = NULL;
 		while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
 		{
 			StripeMetadata *stripe = BuildStripeMetadata(columnarStripes, heapTuple);
 			if (!foundStripeMetadata)
 			{
 				/* first match */
 				foundStripeMetadata = stripe;
 			}
 			else if (lookupMode == FIND_LESS_OR_EQUAL &&
 					 stripe->firstRowNumber > foundStripeMetadata->firstRowNumber)
 			{
 				/*
 				 * Among the stripes with firstRowNumber less-than-or-equal-to given,
 				 * we're looking for the one with the greatest firstRowNumber.
 				 */
 				foundStripeMetadata = stripe;
 			}
 			else if (lookupMode == FIND_GREATER &&
 					 stripe->firstRowNumber < foundStripeMetadata->firstRowNumber)
 			{
 				/*
 				 * Among the stripes with firstRowNumber greater-than given,
 				 * we're looking for the one with the smallest firstRowNumber.
 				 */
 				foundStripeMetadata = stripe;
 			}
 		}
 	}
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	index_close(index, AccessShareLock);
 	table_close(columnarStripes, AccessShareLock);
 	return foundStripeMetadata;
@ -1083,8 +1021,8 @@ CheckStripeMetadataConsistency(StripeMetadata *stripeMetadata)
 /*
 * FindStripeWithHighestRowNumber returns StripeMetadata for the stripe that
- * has the row with highest rowNumber. If given relation is empty, then returns
+ * has the row with highest rowNumber by doing backward index scan on
- * NULL.
+ * stripe_first_row_number_idx. If given relation is empty, then returns NULL.
 */
 StripeMetadata *
 FindStripeWithHighestRowNumber(Relation relation, Snapshot snapshot)
@ -1094,49 +1032,22 @@ FindStripeWithHighestRowNumber(Relation relation, Snapshot snapshot)
 	uint64 storageId = ColumnarStorageGetStorageId(relation, false);
 	ScanKeyData scanKey[1];
 	ScanKeyInit(&scanKey[0], Anum_columnar_stripe_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(storageId));
 	Relation columnarStripes = table_open(ColumnarStripeRelationId(), AccessShareLock);
-
+	Relation index = index_open(ColumnarStripeFirstRowNumberIndexRelationId(),
-	Oid indexId = ColumnarStripeFirstRowNumberIndexRelationId();
+								AccessShareLock);
-	bool indexOk = OidIsValid(indexId);
+	SysScanDesc scanDescriptor = systable_beginscan_ordered(columnarStripes, index,
 	SysScanDesc scanDescriptor = systable_beginscan(columnarStripes, indexId, indexOk,
 															snapshot, 1, scanKey);
-	static bool loggedSlowMetadataAccessWarning = false;
+	HeapTuple heapTuple = systable_getnext_ordered(scanDescriptor, BackwardScanDirection);
 	if (!indexOk && !loggedSlowMetadataAccessWarning)
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING,
 								 "stripe_first_row_number_idx")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	if (indexOk)
 	{
 		/* do one-time fetch using the index */
 		HeapTuple heapTuple = systable_getnext_ordered(scanDescriptor,
 													   BackwardScanDirection);
 	if (HeapTupleIsValid(heapTuple))
 	{
 		stripeWithHighestRowNumber = BuildStripeMetadata(columnarStripes, heapTuple);
 	}
 	}
 	else
 	{
 		HeapTuple heapTuple = NULL;
 		while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
 		{
 			StripeMetadata *stripe = BuildStripeMetadata(columnarStripes, heapTuple);
 			if (!stripeWithHighestRowNumber ||
 				stripe->firstRowNumber > stripeWithHighestRowNumber->firstRowNumber)
 			{
 				/* first or a greater match */
 				stripeWithHighestRowNumber = stripe;
 			}
 		}
 	}
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	index_close(index, AccessShareLock);
 	table_close(columnarStripes, AccessShareLock);
 	return stripeWithHighestRowNumber;
@ -1153,29 +1064,23 @@ ReadChunkGroupRowCounts(uint64 storageId, uint64 stripe, uint32 chunkGroupCount,
 {
 	Oid columnarChunkGroupOid = ColumnarChunkGroupRelationId();
 	Relation columnarChunkGroup = table_open(columnarChunkGroupOid, AccessShareLock);
 	Relation index = index_open(ColumnarChunkGroupIndexRelationId(), AccessShareLock);
 	ScanKeyData scanKey[2];
 	ScanKeyInit(&scanKey[0], Anum_columnar_chunkgroup_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, UInt64GetDatum(storageId));
 	ScanKeyInit(&scanKey[1], Anum_columnar_chunkgroup_stripe,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(stripe));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(stripe));
 	Oid indexId = ColumnarChunkGroupIndexRelationId();
 	bool indexOk = OidIsValid(indexId);
 	SysScanDesc scanDescriptor =
-		systable_beginscan(columnarChunkGroup, indexId, indexOk, snapshot, 2, scanKey);
+		systable_beginscan_ordered(columnarChunkGroup, index, snapshot, 2, scanKey);
 	static bool loggedSlowMetadataAccessWarning = false;
 	if (!indexOk && !loggedSlowMetadataAccessWarning)
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING, "chunk_group_pkey")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	uint32 chunkGroupIndex = 0;
 	HeapTuple heapTuple = NULL;
 	uint32 *chunkGroupRowCounts = palloc0(chunkGroupCount * sizeof(uint32));
-	while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
+	while (HeapTupleIsValid(heapTuple = systable_getnext_ordered(scanDescriptor,
 																 ForwardScanDirection)))
 	{
 		Datum datumArray[Natts_columnar_chunkgroup];
 		bool isNullArray[Natts_columnar_chunkgroup];
@ -1186,16 +1091,24 @@ ReadChunkGroupRowCounts(uint64 storageId, uint64 stripe, uint32 chunkGroupCount,
 		uint32 tupleChunkGroupIndex =
 			DatumGetUInt32(datumArray[Anum_columnar_chunkgroup_chunk - 1]);
-		if (tupleChunkGroupIndex >= chunkGroupCount)
+		if (chunkGroupIndex >= chunkGroupCount ||
 			tupleChunkGroupIndex != chunkGroupIndex)
 		{
 			elog(ERROR, "unexpected chunk group");
 		}
-		chunkGroupRowCounts[tupleChunkGroupIndex] =
+		chunkGroupRowCounts[chunkGroupIndex] =
 			(uint32) DatumGetUInt64(datumArray[Anum_columnar_chunkgroup_row_count - 1]);
 		chunkGroupIndex++;
 	}
-	systable_endscan(scanDescriptor);
+	if (chunkGroupIndex != chunkGroupCount)
 	{
 		elog(ERROR, "unexpected chunk group count");
 	}
 	systable_endscan_ordered(scanDescriptor);
 	index_close(index, AccessShareLock);
 	table_close(columnarChunkGroup, AccessShareLock);
 	return chunkGroupRowCounts;
@ -1248,13 +1161,13 @@ InsertEmptyStripeMetadataRow(uint64 storageId, uint64 stripeId, uint32 columnCou
 /*
- * StripesForRelfilelocator returns a list of StripeMetadata for stripes
+ * StripesForRelfilenode returns a list of StripeMetadata for stripes
 * of the given relfilenode.
 */
 List *
-StripesForRelfilelocator(RelFileLocator relfilelocator)
+StripesForRelfilenode(RelFileNode relfilenode)
 {
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	return ReadDataFileStripeList(storageId, GetTransactionSnapshot());
 }
@ -1269,9 +1182,9 @@ StripesForRelfilelocator(RelFileLocator relfilelocator)
 * returns 0.
 */
 uint64
-GetHighestUsedAddress(RelFileLocator relfilelocator)
+GetHighestUsedAddress(RelFileNode relfilenode)
 {
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	uint64 highestUsedAddress = 0;
 	uint64 highestUsedId = 0;
@ -1385,27 +1298,21 @@ UpdateStripeMetadataRow(uint64 storageId, uint64 stripeId, bool *update,
 	ScanKeyData scanKey[2];
 	ScanKeyInit(&scanKey[0], Anum_columnar_stripe_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(storageId));
 	ScanKeyInit(&scanKey[1], Anum_columnar_stripe_stripe,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(stripeId));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(stripeId));
 	Oid columnarStripesOid = ColumnarStripeRelationId();
 	Relation columnarStripes = table_open(columnarStripesOid, AccessShareLock);
 	Relation columnarStripePkeyIndex = index_open(ColumnarStripePKeyIndexRelationId(),
 												  AccessShareLock);
-	Oid indexId = ColumnarStripePKeyIndexRelationId();
+	SysScanDesc scanDescriptor = systable_beginscan_ordered(columnarStripes,
-	bool indexOk = OidIsValid(indexId);
+															columnarStripePkeyIndex,
 	SysScanDesc scanDescriptor = systable_beginscan(columnarStripes, indexId, indexOk,
 															&dirtySnapshot, 2, scanKey);
-	static bool loggedSlowMetadataAccessWarning = false;
+	HeapTuple oldTuple = systable_getnext_ordered(scanDescriptor, ForwardScanDirection);
 	if (!indexOk && !loggedSlowMetadataAccessWarning)
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING, "stripe_pkey")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	HeapTuple oldTuple = systable_getnext(scanDescriptor);
 	if (!HeapTupleIsValid(oldTuple))
 	{
 		ereport(ERROR, (errmsg("attempted to modify an unexpected stripe, "
@ -1440,7 +1347,8 @@ UpdateStripeMetadataRow(uint64 storageId, uint64 stripeId, bool *update,
 	CommandCounterIncrement();
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	index_close(columnarStripePkeyIndex, AccessShareLock);
 	table_close(columnarStripes, AccessShareLock);
 	/* return StripeMetadata object built from modified tuple */
@ -1451,10 +1359,6 @@ UpdateStripeMetadataRow(uint64 storageId, uint64 stripeId, bool *update,
 /*
 * ReadDataFileStripeList reads the stripe list for a given storageId
 * in the given snapshot.
 *
 * Doesn't sort the stripes by their ids before returning if
 * stripe_first_row_number_idx is not available --normally can only happen
 * during pg upgrades.
 */
 static List *
 ReadDataFileStripeList(uint64 storageId, Snapshot snapshot)
@ -1464,32 +1368,27 @@ ReadDataFileStripeList(uint64 storageId, Snapshot snapshot)
 	HeapTuple heapTuple;
 	ScanKeyInit(&scanKey[0], Anum_columnar_stripe_storageid,
-				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(storageId));
+				BTEqualStrategyNumber, F_OIDEQ, Int32GetDatum(storageId));
 	Oid columnarStripesOid = ColumnarStripeRelationId();
 	Relation columnarStripes = table_open(columnarStripesOid, AccessShareLock);
 	Relation index = index_open(ColumnarStripeFirstRowNumberIndexRelationId(),
 								AccessShareLock);
-	Oid indexId = ColumnarStripeFirstRowNumberIndexRelationId();
+	SysScanDesc scanDescriptor = systable_beginscan_ordered(columnarStripes, index,
-	bool indexOk = OidIsValid(indexId);
+															snapshot, 1,
-	SysScanDesc scanDescriptor = systable_beginscan(columnarStripes, indexId,
+															scanKey);
 													indexOk, snapshot, 1, scanKey);
-	static bool loggedSlowMetadataAccessWarning = false;
+	while (HeapTupleIsValid(heapTuple = systable_getnext_ordered(scanDescriptor,
-	if (!indexOk && !loggedSlowMetadataAccessWarning)
+																 ForwardScanDirection)))
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING,
 								 "stripe_first_row_number_idx")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
 	{
 		StripeMetadata *stripeMetadata = BuildStripeMetadata(columnarStripes, heapTuple);
 		stripeMetadataList = lappend(stripeMetadataList, stripeMetadata);
 	}
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	index_close(index, AccessShareLock);
 	table_close(columnarStripes, AccessShareLock);
 	return stripeMetadataList;
@ -1552,7 +1451,7 @@ BuildStripeMetadata(Relation columnarStripes, HeapTuple heapTuple)
 * metadata tables.
 */
 void
-DeleteMetadataRows(RelFileLocator relfilelocator)
+DeleteMetadataRows(RelFileNode relfilenode)
 {
 	/*
 	 * During a restore for binary upgrade, metadata tables and indexes may or
@ -1563,7 +1462,7 @@ DeleteMetadataRows(RelFileLocator relfilelocator)
 		return;
 	}
-	uint64 storageId = LookupStorageId(relfilelocator);
+	uint64 storageId = LookupStorageId(relfilenode);
 	DeleteStorageFromColumnarMetadataTable(ColumnarStripeRelationId(),
 										   Anum_columnar_stripe_storageid,
@ -1591,7 +1490,7 @@ DeleteStorageFromColumnarMetadataTable(Oid metadataTableId,
 {
 	ScanKeyData scanKey[1];
 	ScanKeyInit(&scanKey[0], storageIdAtrrNumber, BTEqualStrategyNumber,
-				F_INT8EQ, Int64GetDatum(storageId));
+				F_INT8EQ, UInt64GetDatum(storageId));
 	Relation metadataTable = try_relation_open(metadataTableId, AccessShareLock);
 	if (metadataTable == NULL)
@ -1600,30 +1499,25 @@ DeleteStorageFromColumnarMetadataTable(Oid metadataTableId,
 		return;
 	}
-	bool indexOk = OidIsValid(storageIdIndexId);
+	Relation index = index_open(storageIdIndexId, AccessShareLock);
 	SysScanDesc scanDescriptor = systable_beginscan(metadataTable, storageIdIndexId,
 													indexOk, NULL, 1, scanKey);
-	static bool loggedSlowMetadataAccessWarning = false;
+	SysScanDesc scanDescriptor = systable_beginscan_ordered(metadataTable, index, NULL,
-	if (!indexOk && !loggedSlowMetadataAccessWarning)
+															1, scanKey);
 	{
 		ereport(WARNING, (errmsg(SLOW_METADATA_ACCESS_WARNING,
 								 "on a columnar metadata table")));
 		loggedSlowMetadataAccessWarning = true;
 	}
 	ModifyState *modifyState = StartModifyRelation(metadataTable);
 	HeapTuple heapTuple;
-	while (HeapTupleIsValid(heapTuple = systable_getnext(scanDescriptor)))
+	while (HeapTupleIsValid(heapTuple = systable_getnext_ordered(scanDescriptor,
 																 ForwardScanDirection)))
 	{
 		DeleteTupleAndEnforceConstraints(modifyState, heapTuple);
 	}
-	systable_endscan(scanDescriptor);
+	systable_endscan_ordered(scanDescriptor);
 	FinishModifyRelation(modifyState);
 	index_close(index, AccessShareLock);
 	table_close(metadataTable, AccessShareLock);
 }
@ -1636,8 +1530,12 @@ StartModifyRelation(Relation rel)
 {
 	EState *estate = create_estate_for_relation(rel);
 #if PG_VERSION_NUM >= PG_VERSION_14
 	ResultRelInfo *resultRelInfo = makeNode(ResultRelInfo);
 	InitResultRelInfo(resultRelInfo, rel, 1, NULL, 0);
 #else
 	ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
 #endif
 	/* ExecSimpleRelationInsert, ... require caller to open indexes */
 	ExecOpenIndices(resultRelInfo, false);
@ -1667,7 +1565,7 @@ InsertTupleAndEnforceConstraints(ModifyState *state, Datum *values, bool *nulls)
 	ExecStoreHeapTuple(tuple, slot, false);
 	/* use ExecSimpleRelationInsert to enforce constraints */
-	ExecSimpleRelationInsert(state->resultRelInfo, state->estate, slot);
+	ExecSimpleRelationInsert_compat(state->resultRelInfo, state->estate, slot);
 }
@ -1685,7 +1583,7 @@ DeleteTupleAndEnforceConstraints(ModifyState *state, HeapTuple heapTuple)
 	simple_heap_delete(state->rel, tid);
 	/* execute AFTER ROW DELETE Triggers to enforce constraints */
-	ExecARDeleteTriggers(estate, resultRelInfo, tid, NULL, NULL, false);
+	ExecARDeleteTriggers_compat(estate, resultRelInfo, tid, NULL, NULL, false);
 }
@ -1698,8 +1596,12 @@ FinishModifyRelation(ModifyState *state)
 	ExecCloseIndices(state->resultRelInfo);
 	AfterTriggerEndQuery(state->estate);
 #if PG_VERSION_NUM >= PG_VERSION_14
 	ExecCloseResultRelations(state->estate);
 	ExecCloseRangeTableRelations(state->estate);
 #else
 	ExecCleanUpTriggerState(state->estate);
 #endif
 	ExecResetTupleTable(state->estate->es_tupleTable, false);
 	FreeExecutorState(state->estate);
@ -1726,13 +1628,15 @@ create_estate_for_relation(Relation rel)
 	rte->relid = RelationGetRelid(rel);
 	rte->relkind = rel->rd_rel->relkind;
 	rte->rellockmode = AccessShareLock;
 #if PG_VERSION_NUM >= PG_VERSION_16
 	List *perminfos = NIL;
 	addRTEPermissionInfo(&perminfos, rte);
 	ExecInitRangeTable(estate, list_make1(rte), perminfos);
 #else
 	ExecInitRangeTable(estate, list_make1(rte));
 #if PG_VERSION_NUM < PG_VERSION_14
 	ResultRelInfo *resultRelInfo = makeNode(ResultRelInfo);
 	InitResultRelInfo(resultRelInfo, rel, 1, NULL, 0);
 	estate->es_result_relations = resultRelInfo;
 	estate->es_num_result_relations = 1;
 	estate->es_result_relation_info = resultRelInfo;
 #endif
 	estate->es_output_cid = GetCurrentCommandId(true);
@ -1746,9 +1650,6 @@ create_estate_for_relation(Relation rel)
 /*
 * DatumToBytea serializes a datum into a bytea value.
 *
 * Since we don't want to limit datum size to RSIZE_MAX unnecessarily,
 * we use memcpy instead of memcpy_s several places in this function.
 */
 static bytea *
 DatumToBytea(Datum value, Form_pg_attribute attrForm)
@ -1765,16 +1666,19 @@ DatumToBytea(Datum value, Form_pg_attribute attrForm)
 			Datum tmp;
 			store_att_byval(&tmp, value, attrForm->attlen);
-			memcpy(VARDATA(result), &tmp, attrForm->attlen); /* IGNORE-BANNED */
+			memcpy_s(VARDATA(result), datumLength + VARHDRSZ,
 					 &tmp, attrForm->attlen);
 		}
 		else
 		{
-			memcpy(VARDATA(result), DatumGetPointer(value), attrForm->attlen); /* IGNORE-BANNED */
+			memcpy_s(VARDATA(result), datumLength + VARHDRSZ,
 					 DatumGetPointer(value), attrForm->attlen);
 		}
 	}
 	else
 	{
-		memcpy(VARDATA(result), DatumGetPointer(value), datumLength); /* IGNORE-BANNED */
+		memcpy_s(VARDATA(result), datumLength + VARHDRSZ,
 				 DatumGetPointer(value), datumLength);
 	}
 	return result;
@ -1793,12 +1697,8 @@ ByteaToDatum(bytea *bytes, Form_pg_attribute attrForm)
 	 * after the byteaDatum is freed.
 	 */
 	char *binaryDataCopy = palloc0(VARSIZE_ANY_EXHDR(bytes));
-
+	memcpy_s(binaryDataCopy, VARSIZE_ANY_EXHDR(bytes),
-	/*
+			 VARDATA_ANY(bytes), VARSIZE_ANY_EXHDR(bytes));
 	 * We use IGNORE-BANNED here since we don't want to limit datum size to
 	 * RSIZE_MAX unnecessarily.
 	 */
 	memcpy(binaryDataCopy, VARDATA_ANY(bytes), VARSIZE_ANY_EXHDR(bytes)); /* IGNORE-BANNED */
 	return fetch_att(binaryDataCopy, attrForm->attbyval, attrForm->attlen);
 }
@ -1937,11 +1837,10 @@ ColumnarNamespaceId(void)
 * false if the relation doesn't have a meta page yet.
 */
 static uint64
-LookupStorageId(RelFileLocator relfilelocator)
+LookupStorageId(RelFileNode relfilenode)
 {
-	Oid relationId = RelidByRelfilenumber(RelationTablespace_compat(relfilelocator),
+	Oid relationId = RelidByRelfilenode(relfilenode.spcNode,
-										  RelationPhysicalIdentifierNumber_compat(
+										relfilenode.relNode);
 											  relfilelocator));
 	Relation relation = relation_open(relationId, AccessShareLock);
 	uint64 storageId = ColumnarStorageGetStorageId(relation, false);
@ -1972,7 +1871,7 @@ columnar_relation_storageid(PG_FUNCTION_ARGS)
 	Oid relationId = PG_GETARG_OID(0);
 	Relation relation = relation_open(relationId, AccessShareLock);
-	if (!object_ownercheck(RelationRelationId, relationId, GetUserId()))
+	if (!pg_class_ownercheck(relationId, GetUserId()))
 	{
 		aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE,
 					   get_rel_name(relationId));
@ -2041,7 +1940,7 @@ GetHighestUsedRowNumber(uint64 storageId)
 	List *stripeMetadataList = ReadDataFileStripeList(storageId,
 													  GetTransactionSnapshot());
 	StripeMetadata *stripeMetadata = NULL;
-	foreach_declared_ptr(stripeMetadata, stripeMetadataList)
+	foreach_ptr(stripeMetadata, stripeMetadataList)
 	{
 		highestRowNumber = Max(highestRowNumber,
 							   StripeGetHighestRowNumber(stripeMetadata));
--- a/src/backend/columnar/columnar_reader.c
+++ b/src/backend/columnar/columnar_reader.c
@ -22,15 +22,16 @@
 #include "access/xact.h"
 #include "catalog/pg_am.h"
 #include "commands/defrem.h"
 #include "distributed/listutils.h"
 #include "nodes/makefuncs.h"
 #include "nodes/nodeFuncs.h"
 #include "optimizer/clauses.h"
 #include "optimizer/optimizer.h"
 #include "optimizer/clauses.h"
 #include "optimizer/restrictinfo.h"
 #include "storage/fd.h"
 #include "utils/guc.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/lsyscache.h"
 #include "utils/rel.h"
 #include "columnar/columnar.h"
@ -38,8 +39,6 @@
 #include "columnar/columnar_tableam.h"
 #include "columnar/columnar_version_compat.h"
 #include "distributed/listutils.h"
 #define UNEXPECTED_STRIPE_READ_ERR_MSG \
 	"attempted to read an unexpected stripe while reading columnar " \
 	"table %s, stripe with id=" UINT64_FORMAT " is not flushed"
@ -255,9 +254,8 @@ ColumnarReadFlushPendingWrites(ColumnarReadState *readState)
 {
 	Assert(!readState->snapshotRegisteredByUs);
-	RelFileNumber relfilenumber = RelationPhysicalIdentifierNumber_compat(
+	Oid relfilenode = readState->relation->rd_node.relNode;
-		RelationPhysicalIdentifier_compat(readState->relation));
+	FlushWriteStateForRelfilenode(relfilenode, GetCurrentSubTransactionId());
 	FlushWriteStateForRelfilenumber(relfilenumber, GetCurrentSubTransactionId());
 	if (readState->snapshot == InvalidSnapshot || !IsMVCCSnapshot(readState->snapshot))
 	{
@ -880,7 +878,7 @@ ReadChunkGroupNextRow(ChunkGroupReadState *chunkGroupReadState, Datum *columnVal
 	memset(columnNulls, true, sizeof(bool) * chunkGroupReadState->columnCount);
 	int attno;
-	foreach_declared_int(attno, chunkGroupReadState->projectedColumnList)
+	foreach_int(attno, chunkGroupReadState->projectedColumnList)
 	{
 		const ChunkData *chunkGroupData = chunkGroupReadState->chunkGroupData;
 		const int rowIndex = chunkGroupReadState->currentRow;
@ -986,8 +984,7 @@ ColumnarTableRowCount(Relation relation)
 {
 	ListCell *stripeMetadataCell = NULL;
 	uint64 totalRowCount = 0;
-	List *stripeList = StripesForRelfilelocator(RelationPhysicalIdentifier_compat(
+	List *stripeList = StripesForRelfilenode(relation->rd_node);
 													relation));
 	foreach(stripeMetadataCell, stripeList)
 	{
@ -1015,8 +1012,7 @@ LoadFilteredStripeBuffers(Relation relation, StripeMetadata *stripeMetadata,
 	bool *projectedColumnMask = ProjectedColumnMask(columnCount, projectedColumnList);
-	StripeSkipList *stripeSkipList = ReadStripeSkipList(RelationPhysicalIdentifier_compat(
+	StripeSkipList *stripeSkipList = ReadStripeSkipList(relation->rd_node,
 															relation),
 														stripeMetadata->id,
 														tupleDescriptor,
 														stripeMetadata->chunkCount,
@ -1489,7 +1485,7 @@ ProjectedColumnMask(uint32 columnCount, List *projectedColumnList)
 	bool *projectedColumnMask = palloc0(columnCount * sizeof(bool));
 	int attno;
-	foreach_declared_int(attno, projectedColumnList)
+	foreach_int(attno, projectedColumnList)
 	{
 		/* attno is 1-indexed; projectedColumnMask is 0-indexed */
 		int columnIndex = attno - 1;
@ -1561,7 +1557,7 @@ DeserializeDatumArray(StringInfo datumBuffer, bool *existsArray, uint32 datumCou
 										   datumTypeLength);
 		currentDatumDataOffset = att_addlength_datum(currentDatumDataOffset,
 													 datumTypeLength,
-													 datumArray[datumIndex]);
+													 currentDatumDataPointer);
 		currentDatumDataOffset = att_align_nominal(currentDatumDataOffset,
 												   datumTypeAlign);
--- a/src/backend/columnar/columnar_storage.c
+++ b/src/backend/columnar/columnar_storage.c
@ -36,11 +36,11 @@
 #include "postgres.h"
 #include "miscadmin.h"
 #include "safe_lib.h"
 #include "access/generic_xlog.h"
 #include "catalog/storage.h"
 #include "miscadmin.h"
 #include "storage/bufmgr.h"
 #include "storage/lmgr.h"
@ -169,11 +169,7 @@ ColumnarStorageInit(SMgrRelation srel, uint64 storageId)
 	}
 	/* create two pages */
 #if PG_VERSION_NUM >= PG_VERSION_16
 	PGIOAlignedBlock block;
 #else
 	PGAlignedBlock block;
 #endif
 	Page page = block.data;
 	/* write metapage */
@ -192,7 +188,7 @@ ColumnarStorageInit(SMgrRelation srel, uint64 storageId)
 			 (char *) &metapage, sizeof(ColumnarMetapage));
 	phdr->pd_lower += sizeof(ColumnarMetapage);
-	log_newpage(RelationPhysicalIdentifierBackend_compat(&srel), MAIN_FORKNUM,
+	log_newpage(&srel->smgr_rnode.node, MAIN_FORKNUM,
 				COLUMNAR_METAPAGE_BLOCKNO, page, true);
 	PageSetChecksumInplace(page, COLUMNAR_METAPAGE_BLOCKNO);
 	smgrextend(srel, MAIN_FORKNUM, COLUMNAR_METAPAGE_BLOCKNO, page, true);
@ -200,7 +196,7 @@ ColumnarStorageInit(SMgrRelation srel, uint64 storageId)
 	/* write empty page */
 	PageInit(page, BLCKSZ, 0);
-	log_newpage(RelationPhysicalIdentifierBackend_compat(&srel), MAIN_FORKNUM,
+	log_newpage(&srel->smgr_rnode.node, MAIN_FORKNUM,
 				COLUMNAR_EMPTY_BLOCKNO, page, true);
 	PageSetChecksumInplace(page, COLUMNAR_EMPTY_BLOCKNO);
 	smgrextend(srel, MAIN_FORKNUM, COLUMNAR_EMPTY_BLOCKNO, page, true);
--- a/src/backend/columnar/columnar_tableam.c
+++ b/src/backend/columnar/columnar_tableam.c
@ -1,38 +1,41 @@
-#include <math.h>
+#include "citus_version.h"
 #include "postgres.h"
-#include "miscadmin.h"
+#include <math.h>
-#include "pgstat.h"
+
-#include "safe_lib.h"
+#include "miscadmin.h"
 #include "access/detoast.h"
 #include "access/genam.h"
 #include "access/heapam.h"
 #include "access/multixact.h"
 #include "access/rewriteheap.h"
 #include "access/tableam.h"
 #include "access/tsmapi.h"
 #include "access/detoast.h"
 #include "access/xact.h"
 #include "catalog/catalog.h"
 #include "catalog/index.h"
 #include "catalog/namespace.h"
 #include "catalog/objectaccess.h"
 #include "catalog/pg_am.h"
 #include "catalog/pg_extension.h"
 #include "catalog/pg_publication.h"
 #include "catalog/pg_trigger.h"
 #include "catalog/pg_extension.h"
 #include "catalog/storage.h"
 #include "catalog/storage_xlog.h"
 #include "commands/defrem.h"
 #include "commands/extension.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
 #include "commands/extension.h"
 #include "executor/executor.h"
 #include "nodes/makefuncs.h"
 #include "optimizer/plancat.h"
 #include "pgstat.h"
 #include "safe_lib.h"
 #include "storage/bufmgr.h"
 #include "storage/bufpage.h"
 #include "storage/bufmgr.h"
 #include "storage/lmgr.h"
 #include "storage/predicate.h"
 #include "storage/procarray.h"
@ -40,22 +43,17 @@
 #include "tcop/utility.h"
 #include "utils/builtins.h"
 #include "utils/fmgroids.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/pg_rusage.h"
 #include "utils/rel.h"
 #include "utils/relcache.h"
 #include "utils/lsyscache.h"
 #include "utils/syscache.h"
 #include "citus_version.h"
 #include "pg_version_compat.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_customscan.h"
 #include "columnar/columnar_storage.h"
 #include "columnar/columnar_tableam.h"
 #include "columnar/columnar_version_compat.h"
 #include "distributed/listutils.h"
 /*
@ -117,7 +115,9 @@ static RangeVar * ColumnarProcessAlterTable(AlterTableStmt *alterTableStmt,
 											List **columnarOptions);
 static void ColumnarProcessUtility(PlannedStmt *pstmt,
 								   const char *queryString,
 #if PG_VERSION_NUM >= PG_VERSION_14
 								   bool readOnlyTree,
 #endif
 								   ProcessUtilityContext context,
 								   ParamListInfo params,
 								   struct QueryEnvironment *queryEnv,
@ -208,8 +208,7 @@ columnar_beginscan_extended(Relation relation, Snapshot snapshot,
 							uint32 flags, Bitmapset *attr_needed, List *scanQual)
 {
 	CheckCitusColumnarVersion(ERROR);
-	RelFileNumber relfilenumber = RelationPhysicalIdentifierNumber_compat(
+	Oid relfilenode = relation->rd_node.relNode;
 		RelationPhysicalIdentifier_compat(relation));
 	/*
 	 * A memory context to use for scan-wide data, including the lazily
@ -239,7 +238,7 @@ columnar_beginscan_extended(Relation relation, Snapshot snapshot,
 	scan->scanQual = copyObject(scanQual);
 	scan->scanContext = scanContext;
-	if (PendingWritesInUpperTransactions(relfilenumber, GetCurrentSubTransactionId()))
+	if (PendingWritesInUpperTransactions(relfilenode, GetCurrentSubTransactionId()))
 	{
 		elog(ERROR,
 			 "cannot read from table when there is unflushed data in upper transactions");
@ -435,9 +434,8 @@ columnar_index_fetch_begin(Relation rel)
 {
 	CheckCitusColumnarVersion(ERROR);
-	RelFileNumber relfilenumber = RelationPhysicalIdentifierNumber_compat(
+	Oid relfilenode = rel->rd_node.relNode;
-		RelationPhysicalIdentifier_compat(rel));
+	if (PendingWritesInUpperTransactions(relfilenode, GetCurrentSubTransactionId()))
 	if (PendingWritesInUpperTransactions(relfilenumber, GetCurrentSubTransactionId()))
 	{
 		/* XXX: maybe we can just flush the data and continue */
 		elog(ERROR, "cannot read from index when there is unflushed data in "
@ -667,6 +665,7 @@ columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot,
 }
 #if PG_VERSION_NUM >= PG_VERSION_14
 static TransactionId
 columnar_index_delete_tuples(Relation rel,
 							 TM_IndexDeleteOp *delstate)
@ -715,6 +714,19 @@ columnar_index_delete_tuples(Relation rel,
 }
 #else
 static TransactionId
 columnar_compute_xid_horizon_for_tuples(Relation rel,
 										ItemPointerData *tids,
 										int nitems)
 {
 	elog(ERROR, "columnar_compute_xid_horizon_for_tuples not implemented");
 }
 #endif
 static void
 columnar_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
 					  int options, BulkInsertState bistate)
@ -727,9 +739,7 @@ columnar_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
 	 */
 	ColumnarWriteState *writeState = columnar_init_write_state(relation,
 															   RelationGetDescr(relation),
 															   slot->tts_tableOid,
 															   GetCurrentSubTransactionId());
 	MemoryContext oldContext = MemoryContextSwitchTo(ColumnarWritePerTupleContext(
 														 writeState));
@ -771,14 +781,8 @@ columnar_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
 {
 	CheckCitusColumnarVersion(ERROR);
 	/*
 	 * The callback to .multi_insert is table_multi_insert() and this is only used for the COPY
 	 * command, so slot[i]->tts_tableoid will always be equal to relation->id. Thus, we can send
 	 * RelationGetRelid(relation) as the tupSlotTableOid
 	 */
 	ColumnarWriteState *writeState = columnar_init_write_state(relation,
 															   RelationGetDescr(relation),
 															   RelationGetRelid(relation),
 															   GetCurrentSubTransactionId());
 	ColumnarCheckLogicalReplication(relation);
@ -819,7 +823,7 @@ static TM_Result
 columnar_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
 					  CommandId cid, Snapshot snapshot, Snapshot crosscheck,
 					  bool wait, TM_FailureData *tmfd,
-					  LockTupleMode *lockmode, TU_UpdateIndexes *update_indexes)
+					  LockTupleMode *lockmode, bool *update_indexes)
 {
 	elog(ERROR, "columnar_tuple_update not implemented");
 }
@ -845,8 +849,8 @@ columnar_finish_bulk_insert(Relation relation, int options)
 static void
-columnar_relation_set_new_filelocator(Relation rel,
+columnar_relation_set_new_filenode(Relation rel,
-									  const RelFileLocator *newrlocator,
+								   const RelFileNode *newrnode,
 								   char persistence,
 								   TransactionId *freezeXid,
 								   MultiXactId *minmulti)
@ -865,19 +869,16 @@ columnar_relation_set_new_filelocator(Relation rel,
 	 * state. If they are equal, this is a new relation object and we don't
 	 * need to clean anything.
 	 */
-	if (RelationPhysicalIdentifierNumber_compat(RelationPhysicalIdentifier_compat(rel)) !=
+	if (rel->rd_node.relNode != newrnode->relNode)
 		RelationPhysicalIdentifierNumberPtr_compat(newrlocator))
 	{
-		MarkRelfilenumberDropped(RelationPhysicalIdentifierNumber_compat(
+		MarkRelfilenodeDropped(rel->rd_node.relNode, GetCurrentSubTransactionId());
 									 RelationPhysicalIdentifier_compat(rel)),
 								 GetCurrentSubTransactionId());
-		DeleteMetadataRows(RelationPhysicalIdentifier_compat(rel));
+		DeleteMetadataRows(rel->rd_node);
 	}
 	*freezeXid = RecentXmin;
 	*minmulti = GetOldestMultiXactId();
-	SMgrRelation srel = RelationCreateStorage(*newrlocator, persistence, true);
+	SMgrRelation srel = RelationCreateStorage_compat(*newrnode, persistence, true);
 	ColumnarStorageInit(srel, ColumnarMetadataNewStorageId());
 	InitColumnarOptions(rel->rd_id);
@ -892,12 +893,12 @@ static void
 columnar_relation_nontransactional_truncate(Relation rel)
 {
 	CheckCitusColumnarVersion(ERROR);
-	RelFileLocator relfilelocator = RelationPhysicalIdentifier_compat(rel);
+	RelFileNode relfilenode = rel->rd_node;
-	NonTransactionDropWriteState(RelationPhysicalIdentifierNumber_compat(relfilelocator));
+	NonTransactionDropWriteState(relfilenode.relNode);
 	/* Delete old relfilenode metadata */
-	DeleteMetadataRows(relfilelocator);
+	DeleteMetadataRows(relfilenode);
 	/*
 	 * No need to set new relfilenode, since the table was created in this
@ -914,7 +915,7 @@ columnar_relation_nontransactional_truncate(Relation rel)
 static void
-columnar_relation_copy_data(Relation rel, const RelFileLocator *newrnode)
+columnar_relation_copy_data(Relation rel, const RelFileNode *newrnode)
 {
 	elog(ERROR, "columnar_relation_copy_data not implemented");
 }
@ -960,8 +961,7 @@ columnar_relation_copy_for_cluster(Relation OldHeap, Relation NewHeap,
 	ColumnarOptions columnarOptions = { 0 };
 	ReadColumnarOptions(OldHeap->rd_id, &columnarOptions);
-	ColumnarWriteState *writeState = ColumnarBeginWrite(RelationPhysicalIdentifier_compat(
+	ColumnarWriteState *writeState = ColumnarBeginWrite(NewHeap->rd_node,
 															NewHeap),
 														columnarOptions,
 														targetDesc);
@ -1036,8 +1036,7 @@ NeededColumnsList(TupleDesc tupdesc, Bitmapset *attr_needed)
 static uint64
 ColumnarTableTupleCount(Relation relation)
 {
-	List *stripeList = StripesForRelfilelocator(RelationPhysicalIdentifier_compat(
+	List *stripeList = StripesForRelfilenode(relation->rd_node);
 													relation));
 	uint64 tupleCount = 0;
 	ListCell *lc = NULL;
@ -1100,38 +1099,12 @@ columnar_vacuum_rel(Relation rel, VacuumParams *params,
 	List *indexList = RelationGetIndexList(rel);
 	int nindexes = list_length(indexList);
 #if PG_VERSION_NUM >= PG_VERSION_16
 	struct VacuumCutoffs cutoffs;
 	vacuum_get_cutoffs(rel, params, &cutoffs);
 	Assert(MultiXactIdPrecedesOrEquals(cutoffs.MultiXactCutoff, cutoffs.OldestMxact));
 	Assert(TransactionIdPrecedesOrEquals(cutoffs.FreezeLimit, cutoffs.OldestXmin));
 	/*
 	 * Columnar storage doesn't hold any transaction IDs, so we can always
 	 * just advance to the most aggressive value.
 	 */
 	TransactionId newRelFrozenXid = cutoffs.OldestXmin;
 	MultiXactId newRelminMxid = cutoffs.OldestMxact;
 	double new_live_tuples = ColumnarTableTupleCount(rel);
 	/* all visible pages are always 0 */
 	BlockNumber new_rel_allvisible = 0;
 	bool frozenxid_updated;
 	bool minmulti_updated;
 	vac_update_relstats(rel, new_rel_pages, new_live_tuples,
 						new_rel_allvisible, nindexes > 0,
 						newRelFrozenXid, newRelminMxid,
 						&frozenxid_updated, &minmulti_updated, false);
 #else
 	TransactionId oldestXmin;
 	TransactionId freezeLimit;
 	MultiXactId multiXactCutoff;
 	/* initialize xids */
-#if (PG_VERSION_NUM >= PG_VERSION_15) && (PG_VERSION_NUM < PG_VERSION_16)
+#if PG_VERSION_NUM >= PG_VERSION_15
 	MultiXactId oldestMxact;
 	vacuum_set_xid_limits(rel,
 						  params->freeze_min_age,
@ -1161,7 +1134,7 @@ columnar_vacuum_rel(Relation rel, VacuumParams *params,
 	 * just advance to the most aggressive value.
 	 */
 	TransactionId newRelFrozenXid = oldestXmin;
-#if (PG_VERSION_NUM >= PG_VERSION_15) && (PG_VERSION_NUM < PG_VERSION_16)
+#if PG_VERSION_NUM >= PG_VERSION_15
 	MultiXactId newRelminMxid = oldestMxact;
 #else
 	MultiXactId newRelminMxid = multiXactCutoff;
@ -1172,7 +1145,7 @@ columnar_vacuum_rel(Relation rel, VacuumParams *params,
 	/* all visible pages are always 0 */
 	BlockNumber new_rel_allvisible = 0;
-#if (PG_VERSION_NUM >= PG_VERSION_15) && (PG_VERSION_NUM < PG_VERSION_16)
+#if PG_VERSION_NUM >= PG_VERSION_15
 	bool frozenxid_updated;
 	bool minmulti_updated;
@ -1184,7 +1157,6 @@ columnar_vacuum_rel(Relation rel, VacuumParams *params,
 	vac_update_relstats(rel, new_rel_pages, new_live_tuples,
 						new_rel_allvisible, nindexes > 0,
 						newRelFrozenXid, newRelminMxid, false);
 #endif
 #endif
 	pgstat_report_vacuum(RelationGetRelid(rel),
@ -1202,7 +1174,7 @@ static void
 LogRelationStats(Relation rel, int elevel)
 {
 	ListCell *stripeMetadataCell = NULL;
-	RelFileLocator relfilelocator = RelationPhysicalIdentifier_compat(rel);
+	RelFileNode relfilenode = rel->rd_node;
 	StringInfo infoBuf = makeStringInfo();
 	int compressionStats[COMPRESSION_COUNT] = { 0 };
@ -1213,13 +1185,13 @@ LogRelationStats(Relation rel, int elevel)
 	uint64 droppedChunksWithData = 0;
 	uint64 totalDecompressedLength = 0;
-	List *stripeList = StripesForRelfilelocator(relfilelocator);
+	List *stripeList = StripesForRelfilenode(relfilenode);
 	int stripeCount = list_length(stripeList);
 	foreach(stripeMetadataCell, stripeList)
 	{
 		StripeMetadata *stripe = lfirst(stripeMetadataCell);
-		StripeSkipList *skiplist = ReadStripeSkipList(relfilelocator, stripe->id,
+		StripeSkipList *skiplist = ReadStripeSkipList(relfilenode, stripe->id,
 													  RelationGetDescr(rel),
 													  stripe->chunkCount,
 													  GetTransactionSnapshot());
@ -1355,8 +1327,7 @@ TruncateColumnar(Relation rel, int elevel)
 	 * new stripes be added beyond highestPhysicalAddress while
 	 * we're truncating.
 	 */
-	uint64 newDataReservation = Max(GetHighestUsedAddress(
+	uint64 newDataReservation = Max(GetHighestUsedAddress(rel->rd_node) + 1,
 										RelationPhysicalIdentifier_compat(rel)) + 1,
 									ColumnarFirstLogicalOffset);
 	BlockNumber old_rel_pages = smgrnblocks(RelationGetSmgr(rel), MAIN_FORKNUM);
@ -1424,32 +1395,15 @@ ConditionalLockRelationWithTimeout(Relation rel, LOCKMODE lockMode, int timeout,
 static bool
-columnar_scan_analyze_next_block(TableScanDesc scan,
+columnar_scan_analyze_next_block(TableScanDesc scan, BlockNumber blockno,
 #if PG_VERSION_NUM >= PG_VERSION_17
 								 ReadStream *stream)
 #else
 								 BlockNumber blockno,
 								 BufferAccessStrategy bstrategy)
 #endif
 {
 	/*
 	 * Our access method is not pages based, i.e. tuples are not confined
 	 * to pages boundaries. So not much to do here. We return true anyway
 	 * so acquire_sample_rows() in analyze.c would call our
 	 * columnar_scan_analyze_next_tuple() callback.
 	 * In PG17, we return false in case there is no buffer left, since
 	 * the outer loop changed in acquire_sample_rows(), and it is
 	 * expected for the scan_analyze_next_block function to check whether
 	 * there are any blocks left in the block sampler.
 	 */
 #if PG_VERSION_NUM >= PG_VERSION_17
 	Buffer buf = read_stream_next_buffer(stream, NULL);
 	if (!BufferIsValid(buf))
 	{
 		return false;
 	}
 	ReleaseBuffer(buf);
 #endif
 	return true;
 }
@ -1522,7 +1476,8 @@ columnar_index_build_range_scan(Relation columnarRelation,
 	if (!IsBootstrapProcessingMode() && !indexInfo->ii_Concurrent)
 	{
 		/* ignore lazy VACUUM's */
-		OldestXmin = GetOldestNonRemovableTransactionId(columnarRelation);
+		OldestXmin = GetOldestNonRemovableTransactionId_compat(columnarRelation,
 															   PROCARRAY_FLAGS_VACUUM);
 	}
 	Snapshot snapshot = { 0 };
@ -1850,7 +1805,7 @@ ColumnarReadMissingRowsIntoIndex(TableScanDesc scan, Relation indexRelation,
 		Relation columnarRelation = scan->rs_rd;
 		IndexUniqueCheck indexUniqueCheck =
 			indexInfo->ii_Unique ? UNIQUE_CHECK_YES : UNIQUE_CHECK_NO;
-		index_insert(indexRelation, indexValues, indexNulls, columnarItemPointer,
+		index_insert_compat(indexRelation, indexValues, indexNulls, columnarItemPointer,
 							columnarRelation, indexUniqueCheck, false, indexInfo);
 		validateIndexState->tups_inserted += 1;
@ -1880,8 +1835,8 @@ TupleSortSkipSmallerItemPointers(Tuplesortstate *tupleSort, ItemPointer targetIt
 		Datum *abbrev = NULL;
 		Datum tsDatum;
 		bool tsDatumIsNull;
-		if (!tuplesort_getdatum_compat(tupleSort, forwardDirection, false,
+		if (!tuplesort_getdatum(tupleSort, forwardDirection, &tsDatum,
-									   &tsDatum, &tsDatumIsNull, abbrev))
+								&tsDatumIsNull, abbrev))
 		{
 			ItemPointerSetInvalid(&tsItemPointerData);
 			break;
@ -2055,7 +2010,7 @@ columnar_tableam_init()
 		&EnableVersionChecksColumnar,
 		true,
 		PGC_USERSET,
-		GUC_NO_SHOW_ALL | GUC_NOT_IN_SAMPLE,
+		GUC_NO_SHOW_ALL,
 		NULL, NULL, NULL);
 }
@ -2122,13 +2077,12 @@ ColumnarTableDropHook(Oid relid)
 		 * tableam tables storage is managed by postgres.
 		 */
 		Relation rel = table_open(relid, AccessExclusiveLock);
-		RelFileLocator relfilelocator = RelationPhysicalIdentifier_compat(rel);
+		RelFileNode relfilenode = rel->rd_node;
-		DeleteMetadataRows(relfilelocator);
+		DeleteMetadataRows(relfilenode);
 		DeleteColumnarTableOptions(rel->rd_id, true);
-		MarkRelfilenumberDropped(RelationPhysicalIdentifierNumber_compat(relfilelocator),
+		MarkRelfilenodeDropped(relfilenode.relNode, GetCurrentSubTransactionId());
 								 GetCurrentSubTransactionId());
 		/* keep the lock since we did physical changes to the relation */
 		table_close(rel, NoLock);
@ -2245,6 +2199,7 @@ ColumnarProcessAlterTable(AlterTableStmt *alterTableStmt, List **columnarOptions
 				columnarRangeVar = alterTableStmt->relation;
 			}
 		}
 #if PG_VERSION_NUM >= PG_VERSION_15
 		else if (alterTableCmd->subtype == AT_SetAccessMethod)
 		{
 			if (columnarRangeVar || *columnarOptions)
@ -2255,15 +2210,14 @@ ColumnarProcessAlterTable(AlterTableStmt *alterTableStmt, List **columnarOptions
 									"Specify SET ACCESS METHOD before storage parameters, or use separate ALTER TABLE commands.")));
 			}
-			destIsColumnar = (strcmp(alterTableCmd->name ? alterTableCmd->name :
+			destIsColumnar = (strcmp(alterTableCmd->name, COLUMNAR_AM_NAME) == 0);
 									 default_table_access_method,
 									 COLUMNAR_AM_NAME) == 0);
 			if (srcIsColumnar && !destIsColumnar)
 			{
 				DeleteColumnarTableOptions(RelationGetRelid(rel), true);
 			}
 		}
 #endif /* PG_VERSION_15 */
 	}
 	relation_close(rel, NoLock);
@ -2278,17 +2232,21 @@ ColumnarProcessAlterTable(AlterTableStmt *alterTableStmt, List **columnarOptions
 static void
 ColumnarProcessUtility(PlannedStmt *pstmt,
 					   const char *queryString,
 #if PG_VERSION_NUM >= PG_VERSION_14
 					   bool readOnlyTree,
 #endif
 					   ProcessUtilityContext context,
 					   ParamListInfo params,
 					   struct QueryEnvironment *queryEnv,
 					   DestReceiver *dest,
 					   QueryCompletion *completionTag)
 {
 #if PG_VERSION_NUM >= PG_VERSION_14
 	if (readOnlyTree)
 	{
 		pstmt = copyObject(pstmt);
 	}
 #endif
 	Node *parsetree = pstmt->utilityStmt;
@ -2405,7 +2363,7 @@ ColumnarProcessUtility(PlannedStmt *pstmt,
 		CheckCitusColumnarAlterExtensionStmt(parsetree);
 	}
-	PrevProcessUtilityHook(pstmt, queryString, false, context,
+	PrevProcessUtilityHook_compat(pstmt, queryString, false, context,
 								  params, queryEnv, dest, completionTag);
 	if (columnarOptions != NIL)
@ -2534,7 +2492,11 @@ static const TableAmRoutine columnar_am_methods = {
 	.tuple_get_latest_tid = columnar_get_latest_tid,
 	.tuple_tid_valid = columnar_tuple_tid_valid,
 	.tuple_satisfies_snapshot = columnar_tuple_satisfies_snapshot,
 #if PG_VERSION_NUM >= PG_VERSION_14
 	.index_delete_tuples = columnar_index_delete_tuples,
 #else
 	.compute_xid_horizon_for_tuples = columnar_compute_xid_horizon_for_tuples,
 #endif
 	.tuple_insert = columnar_tuple_insert,
 	.tuple_insert_speculative = columnar_tuple_insert_speculative,
@ -2545,11 +2507,7 @@ static const TableAmRoutine columnar_am_methods = {
 	.tuple_lock = columnar_tuple_lock,
 	.finish_bulk_insert = columnar_finish_bulk_insert,
-#if PG_VERSION_NUM >= PG_VERSION_16
+	.relation_set_new_filenode = columnar_relation_set_new_filenode,
 	.relation_set_new_filelocator = columnar_relation_set_new_filelocator,
 #else
 	.relation_set_new_filenode = columnar_relation_set_new_filelocator,
 #endif
 	.relation_nontransactional_truncate = columnar_relation_nontransactional_truncate,
 	.relation_copy_data = columnar_relation_copy_data,
 	.relation_copy_for_cluster = columnar_relation_copy_for_cluster,
@ -2610,13 +2568,8 @@ detoast_values(TupleDesc tupleDesc, Datum *orig_values, bool *isnull)
 			if (values == orig_values)
 			{
 				values = palloc(sizeof(Datum) * natts);
-
+				memcpy_s(values, sizeof(Datum) * natts,
-				/*
+						 orig_values, sizeof(Datum) * natts);
 				 * We use IGNORE-BANNED here since we don't want to limit
 				 * size of the buffer that holds the datum array to RSIZE_MAX
 				 * unnecessarily.
 				 */
 				memcpy(values, orig_values, sizeof(Datum) * natts); /* IGNORE-BANNED */
 			}
 			/* will be freed when per-tuple context is reset */
@ -2647,12 +2600,21 @@ ColumnarCheckLogicalReplication(Relation rel)
 		return;
 	}
 #if PG_VERSION_NUM >= PG_VERSION_15
 	{
 		PublicationDesc pubdesc;
 		RelationBuildPublicationDesc(rel, &pubdesc);
 		pubActionInsert = pubdesc.pubactions.pubinsert;
 	}
 #else
 	if (rel->rd_pubactions == NULL)
 	{
 		GetRelationPublicationActions(rel);
 		Assert(rel->rd_pubactions != NULL);
 	}
 	pubActionInsert = rel->rd_pubactions->pubinsert;
 #endif
 	if (pubActionInsert)
 	{
@ -2954,7 +2916,7 @@ MajorVersionsCompatibleColumnar(char *leftVersion, char *rightVersion)
 	}
 	else
 	{
-		rightComparisionLimit = strlen(rightVersion);
+		rightComparisionLimit = strlen(leftVersion);
 	}
 	/* we can error out early if hypens are not in the same position */
@ -3029,8 +2991,6 @@ AvailableExtensionVersionColumnar(void)
 	ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
 					errmsg("citus extension is not found")));
 	return NULL; /* keep compiler happy */
 }
@ -3093,7 +3053,7 @@ DefElem *
 GetExtensionOption(List *extensionOptions, const char *defname)
 {
 	DefElem *defElement = NULL;
-	foreach_declared_ptr(defElement, extensionOptions)
+	foreach_ptr(defElement, extensionOptions)
 	{
 		if (IsA(defElement, DefElem) &&
 			strncmp(defElement->defname, defname, NAMEDATALEN) == 0)
--- a/src/backend/columnar/columnar_writer.c
+++ b/src/backend/columnar/columnar_writer.c
@ -16,37 +16,28 @@
 #include "postgres.h"
 #include "miscadmin.h"
 #include "safe_lib.h"
 #include "access/heapam.h"
 #include "access/nbtree.h"
 #include "catalog/pg_am.h"
 #include "miscadmin.h"
 #include "storage/fd.h"
 #include "storage/smgr.h"
 #include "utils/guc.h"
 #include "utils/memutils.h"
 #include "utils/rel.h"
-
+#include "utils/relfilenodemap.h"
 #include "pg_version_compat.h"
 #include "pg_version_constants.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_storage.h"
 #include "columnar/columnar_version_compat.h"
 #if PG_VERSION_NUM >= PG_VERSION_16
 #include "storage/relfilelocator.h"
 #include "utils/relfilenumbermap.h"
 #else
 #include "utils/relfilenodemap.h"
 #endif
 struct ColumnarWriteState
 {
 	TupleDesc tupleDescriptor;
 	FmgrInfo **comparisonFunctionArray;
-	RelFileLocator relfilelocator;
+	RelFileNode relfilenode;
 	MemoryContext stripeWriteContext;
 	MemoryContext perTupleContext;
@ -93,7 +84,7 @@ static StringInfo CopyStringInfo(StringInfo sourceString);
 * data load operation.
 */
 ColumnarWriteState *
-ColumnarBeginWrite(RelFileLocator relfilelocator,
+ColumnarBeginWrite(RelFileNode relfilenode,
 				   ColumnarOptions options,
 				   TupleDesc tupleDescriptor)
 {
@ -133,7 +124,7 @@ ColumnarBeginWrite(RelFileLocator relfilelocator,
 												options.chunkRowCount);
 	ColumnarWriteState *writeState = palloc0(sizeof(ColumnarWriteState));
-	writeState->relfilelocator = relfilelocator;
+	writeState->relfilenode = relfilenode;
 	writeState->options = options;
 	writeState->tupleDescriptor = CreateTupleDescCopy(tupleDescriptor);
 	writeState->comparisonFunctionArray = comparisonFunctionArray;
@ -183,10 +174,8 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Datum *columnValues, bool *colu
 		writeState->stripeSkipList = stripeSkipList;
 		writeState->compressionBuffer = makeStringInfo();
-		Oid relationId = RelidByRelfilenumber(RelationTablespace_compat(
+		Oid relationId = RelidByRelfilenode(writeState->relfilenode.spcNode,
-												  writeState->relfilelocator),
+											writeState->relfilenode.relNode);
 											  RelationPhysicalIdentifierNumber_compat(
 												  writeState->relfilelocator));
 		Relation relation = relation_open(relationId, NoLock);
 		writeState->emptyStripeReservation =
 			ReserveEmptyStripe(relation, columnCount, chunkRowCount,
@ -404,10 +393,8 @@ FlushStripe(ColumnarWriteState *writeState)
 	elog(DEBUG1, "Flushing Stripe of size %d", stripeBuffers->rowCount);
-	Oid relationId = RelidByRelfilenumber(RelationTablespace_compat(
+	Oid relationId = RelidByRelfilenode(writeState->relfilenode.spcNode,
-											  writeState->relfilelocator),
+										writeState->relfilenode.relNode);
 										  RelationPhysicalIdentifierNumber_compat(
 											  writeState->relfilelocator));
 	Relation relation = relation_open(relationId, NoLock);
 	/*
@ -499,10 +486,10 @@ FlushStripe(ColumnarWriteState *writeState)
 		}
 	}
-	SaveChunkGroups(writeState->relfilelocator,
+	SaveChunkGroups(writeState->relfilenode,
 					stripeMetadata->id,
 					writeState->chunkGroupRowCounts);
-	SaveStripeSkipList(writeState->relfilelocator,
+	SaveStripeSkipList(writeState->relfilenode,
 					   stripeMetadata->id,
 					   stripeSkipList, tupleDescriptor);
@ -544,9 +531,6 @@ SerializeBoolArray(bool *boolArray, uint32 boolArrayLength)
 /*
 * SerializeSingleDatum serializes the given datum value and appends it to the
 * provided string info buffer.
 *
 * Since we don't want to limit datum buffer size to RSIZE_MAX unnecessarily,
 * we use memcpy instead of memcpy_s several places in this function.
 */
 static void
 SerializeSingleDatum(StringInfo datumBuffer, Datum datum, bool datumTypeByValue,
@ -568,13 +552,15 @@ SerializeSingleDatum(StringInfo datumBuffer, Datum datum, bool datumTypeByValue,
 		}
 		else
 		{
-			memcpy(currentDatumDataPointer, DatumGetPointer(datum), datumTypeLength); /* IGNORE-BANNED */
+			memcpy_s(currentDatumDataPointer, datumBuffer->maxlen - datumBuffer->len,
 					 DatumGetPointer(datum), datumTypeLength);
 		}
 	}
 	else
 	{
 		Assert(!datumTypeByValue);
-		memcpy(currentDatumDataPointer, DatumGetPointer(datum), datumLength); /* IGNORE-BANNED */
+		memcpy_s(currentDatumDataPointer, datumBuffer->maxlen - datumBuffer->len,
 				 DatumGetPointer(datum), datumLength);
 	}
 	datumBuffer->len += datumLengthAligned;
@ -728,12 +714,7 @@ DatumCopy(Datum datum, bool datumTypeByValue, int datumTypeLength)
 	{
 		uint32 datumLength = att_addlength_datum(0, datumTypeLength, datum);
 		char *datumData = palloc0(datumLength);
-
+		memcpy_s(datumData, datumLength, DatumGetPointer(datum), datumLength);
 		/*
 		 * We use IGNORE-BANNED here since we don't want to limit datum size to
 		 * RSIZE_MAX unnecessarily.
 		 */
 		memcpy(datumData, DatumGetPointer(datum), datumLength); /* IGNORE-BANNED */
 		datumCopy = PointerGetDatum(datumData);
 	}
@ -756,12 +737,8 @@ CopyStringInfo(StringInfo sourceString)
 		targetString->data = palloc0(sourceString->len);
 		targetString->len = sourceString->len;
 		targetString->maxlen = sourceString->len;
-
+		memcpy_s(targetString->data, sourceString->len,
-		/*
+				 sourceString->data, sourceString->len);
 		 * We use IGNORE-BANNED here since we don't want to limit string
 		 * buffer size to RSIZE_MAX unnecessarily.
 		 */
 		memcpy(targetString->data, sourceString->data, sourceString->len); /* IGNORE-BANNED */
 	}
 	return targetString;
--- a/src/backend/columnar/sql/citus_columnar--11.1-1--11.2-1.sql
+++ b/src/backend/columnar/sql/citus_columnar--11.1-1--11.2-1.sql
@ -1,19 +0,0 @@
 -- citus_columnar--11.1-1--11.2-1
 #include "udfs/columnar_ensure_am_depends_catalog/11.2-1.sql"
 DELETE FROM pg_depend
 WHERE classid = 'pg_am'::regclass::oid
    AND objid IN (select oid from pg_am where amname = 'columnar')
    AND objsubid = 0
    AND refclassid = 'pg_class'::regclass::oid
    AND refobjid IN (
        'columnar_internal.stripe_first_row_number_idx'::regclass::oid,
        'columnar_internal.chunk_group_pkey'::regclass::oid,
        'columnar_internal.chunk_pkey'::regclass::oid,
        'columnar_internal.options_pkey'::regclass::oid,
        'columnar_internal.stripe_first_row_number_idx'::regclass::oid,
        'columnar_internal.stripe_pkey'::regclass::oid
    )
    AND refobjsubid = 0
    AND deptype = 'n';
--- a/src/backend/columnar/sql/citus_columnar--11.2-1--11.3-1.sql
+++ b/src/backend/columnar/sql/citus_columnar--11.2-1--11.3-1.sql
@ -1 +0,0 @@
 -- citus_columnar--11.2-1--11.3-1
--- a/src/backend/columnar/sql/citus_columnar--11.3-1--12.2-1.sql
+++ b/src/backend/columnar/sql/citus_columnar--11.3-1--12.2-1.sql
@ -1 +0,0 @@
 -- citus_columnar--11.3-1--12.2-1
--- a/src/backend/columnar/sql/columnar--10.2-2--10.2-3.sql
+++ b/src/backend/columnar/sql/columnar--10.2-2--10.2-3.sql
@ -10,17 +10,6 @@
 --
 -- To do that, drop stripe_first_row_number_idx and create a unique
 -- constraint with the same name to keep the code change at minimum.
 --
 -- If we have a pg_depend entry for this index, we can not drop it as
 -- the extension depends on it. Remove the pg_depend entry if it exists.
 DELETE FROM pg_depend
 WHERE classid = 'pg_am'::regclass::oid
    AND objid IN (select oid from pg_am where amname = 'columnar')
    AND objsubid = 0
    AND refclassid = 'pg_class'::regclass::oid
    AND refobjid = 'columnar.stripe_first_row_number_idx'::regclass::oid
    AND refobjsubid = 0
    AND deptype = 'n';
 DROP INDEX columnar.stripe_first_row_number_idx;
 ALTER TABLE columnar.stripe ADD CONSTRAINT stripe_first_row_number_idx
 UNIQUE (storage_id, first_row_number);
--- a/src/backend/columnar/sql/downgrades/citus_columnar--11.2-1--11.1-1.sql
+++ b/src/backend/columnar/sql/downgrades/citus_columnar--11.2-1--11.1-1.sql
@ -1,4 +0,0 @@
 -- citus_columnar--11.2-1--11.1-1
 -- Note that we intentionally do not re-insert the pg_depend records that we
 -- deleted via citus_columnar--11.1-1--11.2-1.sql.
--- a/src/backend/columnar/sql/downgrades/citus_columnar--11.3-1--11.2-1.sql
+++ b/src/backend/columnar/sql/downgrades/citus_columnar--11.3-1--11.2-1.sql
@ -1 +0,0 @@
 -- citus_columnar--11.3-1--11.2-1
--- a/src/backend/columnar/sql/downgrades/citus_columnar--12.2-1--11.3-1.sql
+++ b/src/backend/columnar/sql/downgrades/citus_columnar--12.2-1--11.3-1.sql
@ -1 +0,0 @@
 -- citus_columnar--12.2-1--11.3-1
--- a/src/backend/columnar/sql/downgrades/columnar--10.2-1--10.1-1.sql
+++ b/src/backend/columnar/sql/downgrades/columnar--10.2-1--10.1-1.sql
@ -8,16 +8,5 @@ DROP FUNCTION citus_internal.upgrade_columnar_storage(regclass);
 DROP FUNCTION citus_internal.downgrade_columnar_storage(regclass);
 -- drop "first_row_number" column and the index defined on it
 --
 -- If we have a pg_depend entry for this index, we can not drop it as
 -- the extension depends on it. Remove the pg_depend entry if it exists.
 DELETE FROM pg_depend
 WHERE classid = 'pg_am'::regclass::oid
    AND objid IN (select oid from pg_am where amname = 'columnar')
    AND objsubid = 0
    AND refclassid = 'pg_class'::regclass::oid
    AND refobjid = 'columnar.stripe_first_row_number_idx'::regclass::oid
    AND refobjsubid = 0
    AND deptype = 'n';
 DROP INDEX columnar.stripe_first_row_number_idx;
 ALTER TABLE columnar.stripe DROP COLUMN first_row_number;
--- a/src/backend/columnar/sql/downgrades/columnar--10.2-3--10.2-2.sql
+++ b/src/backend/columnar/sql/downgrades/columnar--10.2-3--10.2-2.sql
@ -1,14 +1,4 @@
 -- columnar--10.2-3--10.2-2.sql
--
+
 -- If we have a pg_depend entry for this index, we can not drop it as
 -- the extension depends on it. Remove the pg_depend entry if it exists.
 DELETE FROM pg_depend
 WHERE classid = 'pg_am'::regclass::oid
    AND objid IN (select oid from pg_am where amname = 'columnar')
    AND objsubid = 0
    AND refclassid = 'pg_class'::regclass::oid
    AND refobjid = 'columnar.stripe_first_row_number_idx'::regclass::oid
    AND refobjsubid = 0
    AND deptype = 'n';
 ALTER TABLE columnar.stripe DROP CONSTRAINT stripe_first_row_number_idx;
 CREATE INDEX stripe_first_row_number_idx ON columnar.stripe USING BTREE(storage_id, first_row_number);
--- a/src/backend/columnar/sql/udfs/columnar_ensure_am_depends_catalog/11.2-1.sql
+++ b/src/backend/columnar/sql/udfs/columnar_ensure_am_depends_catalog/11.2-1.sql
@ -1,43 +0,0 @@
 CREATE OR REPLACE FUNCTION columnar_internal.columnar_ensure_am_depends_catalog()
  RETURNS void
  LANGUAGE plpgsql
  SET search_path = pg_catalog
 AS $func$
 BEGIN
  INSERT INTO pg_depend
  WITH columnar_schema_members(relid) AS (
    SELECT pg_class.oid AS relid FROM pg_class
      WHERE relnamespace =
            COALESCE(
 	       (SELECT pg_namespace.oid FROM pg_namespace WHERE nspname = 'columnar_internal'),
 	       (SELECT pg_namespace.oid FROM pg_namespace WHERE nspname = 'columnar')
 	    )
        AND relname IN ('chunk',
                        'chunk_group',
                        'options',
                        'storageid_seq',
                        'stripe')
  )
  SELECT -- Define a dependency edge from "columnar table access method" ..
         'pg_am'::regclass::oid as classid,
         (select oid from pg_am where amname = 'columnar') as objid,
         0 as objsubid,
         -- ... to some objects registered as regclass and that lives in
         -- "columnar" schema. That contains catalog tables and the sequences
         -- created in "columnar" schema.
         --
         -- Given the possibility of user might have created their own objects
         -- in columnar schema, we explicitly specify list of objects that we
         -- are interested in.
         'pg_class'::regclass::oid as refclassid,
         columnar_schema_members.relid as refobjid,
         0 as refobjsubid,
         'n' as deptype
  FROM columnar_schema_members
  -- Avoid inserting duplicate entries into pg_depend.
  EXCEPT TABLE pg_depend;
 END;
 $func$;
 COMMENT ON FUNCTION columnar_internal.columnar_ensure_am_depends_catalog()
  IS 'internal function responsible for creating dependencies from columnar '
     'table access method to the rel objects in columnar schema';
--- a/src/backend/columnar/sql/udfs/columnar_ensure_am_depends_catalog/latest.sql
+++ b/src/backend/columnar/sql/udfs/columnar_ensure_am_depends_catalog/latest.sql
@ -1,4 +1,4 @@
-CREATE OR REPLACE FUNCTION columnar_internal.columnar_ensure_am_depends_catalog()
+CREATE OR REPLACE FUNCTION citus_internal.columnar_ensure_am_depends_catalog()
  RETURNS void
  LANGUAGE plpgsql
  SET search_path = pg_catalog
@ -14,17 +14,22 @@ BEGIN
 	    )
        AND relname IN ('chunk',
                        'chunk_group',
                        'chunk_group_pkey',
                        'chunk_pkey',
                        'options',
                        'options_pkey',
                        'storageid_seq',
-                        'stripe')
+                        'stripe',
                        'stripe_first_row_number_idx',
                        'stripe_pkey')
  )
  SELECT -- Define a dependency edge from "columnar table access method" ..
         'pg_am'::regclass::oid as classid,
         (select oid from pg_am where amname = 'columnar') as objid,
         0 as objsubid,
-         -- ... to some objects registered as regclass and that lives in
+         -- ... to each object that is registered to pg_class and that lives
-         -- "columnar" schema. That contains catalog tables and the sequences
+         -- in "columnar" schema. That contains catalog tables, indexes
-         -- created in "columnar" schema.
+         -- created on them and the sequences created in "columnar" schema.
         --
         -- Given the possibility of user might have created their own objects
         -- in columnar schema, we explicitly specify list of objects that we
@ -38,6 +43,6 @@ BEGIN
  EXCEPT TABLE pg_depend;
 END;
 $func$;
-COMMENT ON FUNCTION columnar_internal.columnar_ensure_am_depends_catalog()
+COMMENT ON FUNCTION citus_internal.columnar_ensure_am_depends_catalog()
  IS 'internal function responsible for creating dependencies from columnar '
     'table access method to the rel objects in columnar schema';
--- a/src/backend/columnar/write_state_management.c
+++ b/src/backend/columnar/write_state_management.c
@ -1,17 +1,21 @@
 #include "citus_version.h"
 #include "postgres.h"
 #include "columnar/columnar.h"
 #include <math.h>
 #include "postgres.h"
 #include "miscadmin.h"
 #include "pgstat.h"
 #include "access/genam.h"
 #include "access/heapam.h"
 #include "access/heaptoast.h"
 #include "access/multixact.h"
 #include "access/rewriteheap.h"
 #include "access/tsmapi.h"
 #include "access/heaptoast.h"
 #include "common/hashfn.h"
 #include "access/xact.h"
 #include "catalog/catalog.h"
 #include "catalog/index.h"
@ -22,12 +26,13 @@
 #include "catalog/storage_xlog.h"
 #include "commands/progress.h"
 #include "commands/vacuum.h"
 #include "common/hashfn.h"
 #include "executor/executor.h"
 #include "nodes/makefuncs.h"
 #include "optimizer/plancat.h"
 #include "pgstat.h"
 #include "storage/bufmgr.h"
 #include "storage/bufpage.h"
 #include "storage/bufmgr.h"
 #include "storage/lmgr.h"
 #include "storage/predicate.h"
 #include "storage/procarray.h"
@ -38,10 +43,6 @@
 #include "utils/rel.h"
 #include "utils/syscache.h"
 #include "citus_version.h"
 #include "pg_version_compat.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_customscan.h"
 #include "columnar/columnar_tableam.h"
 #include "columnar/columnar_version_compat.h"
@ -76,7 +77,7 @@ typedef struct SubXidWriteState
 typedef struct WriteStateMapEntry
 {
 	/* key of the entry */
-	RelFileNumber relfilenumber;
+	Oid relfilenode;
 	/*
 	 * If a table is dropped, we set dropped to true and set dropSubXid to the
@ -112,7 +113,6 @@ CleanupWriteStateMap(void *arg)
 ColumnarWriteState *
 columnar_init_write_state(Relation relation, TupleDesc tupdesc,
 						  Oid tupSlotRelationId,
 						  SubTransactionId currentSubXid)
 {
 	bool found;
@ -131,7 +131,7 @@ columnar_init_write_state(Relation relation, TupleDesc tupdesc,
 		HASHCTL info;
 		uint32 hashFlags = (HASH_ELEM | HASH_FUNCTION | HASH_CONTEXT);
 		memset(&info, 0, sizeof(info));
-		info.keysize = sizeof(RelFileNumber);
+		info.keysize = sizeof(Oid);
 		info.hash = oid_hash;
 		info.entrysize = sizeof(WriteStateMapEntry);
 		info.hcxt = WriteStateContext;
@ -145,10 +145,7 @@ columnar_init_write_state(Relation relation, TupleDesc tupdesc,
 		MemoryContextRegisterResetCallback(WriteStateContext, &cleanupCallback);
 	}
-	WriteStateMapEntry *hashEntry = hash_search(WriteStateMap,
+	WriteStateMapEntry *hashEntry = hash_search(WriteStateMap, &relation->rd_node.relNode,
 												&RelationPhysicalIdentifierNumber_compat(
 													RelationPhysicalIdentifier_compat(
 														relation)),
 												HASH_ENTER, &found);
 	if (!found)
 	{
@ -179,20 +176,10 @@ columnar_init_write_state(Relation relation, TupleDesc tupdesc,
 	MemoryContext oldContext = MemoryContextSwitchTo(WriteStateContext);
 	ColumnarOptions columnarOptions = { 0 };
-
+	ReadColumnarOptions(relation->rd_id, &columnarOptions);
 	/*
 	 * In case of a table rewrite, we need to fetch table options based on the
 	 * relation id of the source tuple slot.
 	 *
 	 * For this reason, we always pass tupSlotRelationId here; which should be
 	 * same as the target table if the write operation is not related to a table
 	 * rewrite etc.
 	 */
 	ReadColumnarOptions(tupSlotRelationId, &columnarOptions);
 	SubXidWriteState *stackEntry = palloc0(sizeof(SubXidWriteState));
-	stackEntry->writeState = ColumnarBeginWrite(RelationPhysicalIdentifier_compat(
+	stackEntry->writeState = ColumnarBeginWrite(relation->rd_node,
 													relation),
 												columnarOptions,
 												tupdesc);
 	stackEntry->subXid = currentSubXid;
@ -209,16 +196,14 @@ columnar_init_write_state(Relation relation, TupleDesc tupdesc,
 * Flushes pending writes for given relfilenode in the given subtransaction.
 */
 void
-FlushWriteStateForRelfilenumber(RelFileNumber relfilenumber,
+FlushWriteStateForRelfilenode(Oid relfilenode, SubTransactionId currentSubXid)
 								SubTransactionId currentSubXid)
 {
 	if (WriteStateMap == NULL)
 	{
 		return;
 	}
-	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenumber, HASH_FIND,
+	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenode, HASH_FIND, NULL);
 											NULL);
 	Assert(!entry || !entry->dropped);
@ -325,14 +310,14 @@ DiscardWriteStateForAllRels(SubTransactionId currentSubXid, SubTransactionId par
 * Called when the given relfilenode is dropped.
 */
 void
-MarkRelfilenumberDropped(RelFileNumber relfilenumber, SubTransactionId currentSubXid)
+MarkRelfilenodeDropped(Oid relfilenode, SubTransactionId currentSubXid)
 {
 	if (WriteStateMap == NULL)
 	{
 		return;
 	}
-	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenumber, HASH_FIND,
+	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenode, HASH_FIND,
 											NULL);
 	if (!entry || entry->dropped)
 	{
@ -348,11 +333,11 @@ MarkRelfilenumberDropped(RelFileNumber relfilenumber, SubTransactionId currentSu
 * Called when the given relfilenode is dropped in non-transactional TRUNCATE.
 */
 void
-NonTransactionDropWriteState(RelFileNumber relfilenumber)
+NonTransactionDropWriteState(Oid relfilenode)
 {
 	if (WriteStateMap)
 	{
-		hash_search(WriteStateMap, &relfilenumber, HASH_REMOVE, false);
+		hash_search(WriteStateMap, &relfilenode, HASH_REMOVE, false);
 	}
 }
@ -361,16 +346,14 @@ NonTransactionDropWriteState(RelFileNumber relfilenumber)
 * Returns true if there are any pending writes in upper transactions.
 */
 bool
-PendingWritesInUpperTransactions(RelFileNumber relfilenumber,
+PendingWritesInUpperTransactions(Oid relfilenode, SubTransactionId currentSubXid)
 								 SubTransactionId currentSubXid)
 {
 	if (WriteStateMap == NULL)
 	{
 		return false;
 	}
-	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenumber, HASH_FIND,
+	WriteStateMapEntry *entry = hash_search(WriteStateMap, &relfilenode, HASH_FIND, NULL);
 											NULL);
 	if (entry && entry->writeStateStack != NULL)
 	{
--- a/src/backend/distributed/Makefile
+++ b/src/backend/distributed/Makefile
@ -18,7 +18,7 @@ generated_downgrade_sql_files += $(patsubst %,$(citus_abs_srcdir)/build/sql/%,$(
 DATA_built = $(generated_sql_files)
 # directories with source files
-SUBDIRS = . commands connection ddl deparser executor metadata operations planner progress relay safeclib shardsplit stats test transaction utils worker clock
+SUBDIRS = . commands connection ddl deparser executor metadata operations planner progress relay safeclib shardsplit test transaction utils worker
 # enterprise modules
 SUBDIRS += replication
@ -32,12 +32,7 @@ OBJS += \
 	$(patsubst $(citus_abs_srcdir)/%.c,%.o,$(foreach dir,$(SUBDIRS), $(sort $(wildcard $(citus_abs_srcdir)/$(dir)/*.c))))
 # be explicit about the default target
-.PHONY: cdc
+all:
 all: cdc
 cdc:
 	$(MAKE) -C cdc all
 NO_PGXS = 1
@ -84,22 +79,13 @@ ifneq (,$(SQL_Po_files))
 include $(SQL_Po_files)
 endif
-
+.PHONY: clean-full install install-downgrades install-all
 .PHONY: clean-full install install-downgrades install-all install-cdc clean-cdc
 clean: clean-cdc
 clean-cdc:
 	$(MAKE) -C cdc clean
 cleanup-before-install:
 	rm -f $(DESTDIR)$(datadir)/$(datamoduledir)/citus.control
 	rm -f $(DESTDIR)$(datadir)/$(datamoduledir)/citus--*
-install: cleanup-before-install install-cdc
+install: cleanup-before-install
 install-cdc:
 	$(MAKE) -C cdc install
 # install and install-downgrades should be run sequentially
 install-all: install
@ -110,5 +96,4 @@ install-downgrades: $(generated_downgrade_sql_files)
 clean-full:
 	$(MAKE) clean
 	$(MAKE) -C cdc clean-full
 	rm -rf $(safestringlib_builddir)
--- a/src/backend/distributed/README.md
+++ b/src/backend/distributed/README.md
--- a/src/backend/distributed/cdc/Makefile
+++ b/src/backend/distributed/cdc/Makefile
@ -1,45 +0,0 @@
 citus_top_builddir = ../../../..
 include $(citus_top_builddir)/Makefile.global
 citus_subdir = src/backend/distributed/cdc
 SRC_DIR = $(citus_abs_top_srcdir)/$(citus_subdir)
 #List of supported based decoders. Add new decoders here.
 cdc_base_decoders :=pgoutput wal2json
 all: build-cdc-decoders
 copy-decoder-files-to-build-dir:
 	$(eval DECODER_BUILD_DIR=build-cdc-$(DECODER))
 	mkdir -p $(DECODER_BUILD_DIR)
 	@for file in $(SRC_DIR)/*.c $(SRC_DIR)/*.h; do \
 		if [ -f $$file ]; then \
 			if [ -f $(DECODER_BUILD_DIR)/$$(basename $$file) ]; then \
 				if ! diff -q $$file $(DECODER_BUILD_DIR)/$$(basename $$file); then \
 					cp $$file $(DECODER_BUILD_DIR)/$$(basename $$file); \
 				fi \
 			else \
 				cp $$file $(DECODER_BUILD_DIR)/$$(basename $$file); \
 			fi \
 		fi \
 	done
 	cp $(SRC_DIR)/Makefile.decoder $(DECODER_BUILD_DIR)/Makefile
 build-cdc-decoders:
 	$(foreach base_decoder,$(cdc_base_decoders),$(MAKE) DECODER=$(base_decoder) build-cdc-decoder;)
 install-cdc-decoders:
 	$(foreach base_decoder,$(cdc_base_decoders),$(MAKE) DECODER=$(base_decoder) -C build-cdc-$(base_decoder) install;)
 clean-cdc-decoders:
 	$(foreach base_decoder,$(cdc_base_decoders),rm -rf build-cdc-$(base_decoder);)
 build-cdc-decoder:
 	$(MAKE) DECODER=$(DECODER) copy-decoder-files-to-build-dir
 	$(MAKE) DECODER=$(DECODER) -C build-cdc-$(DECODER)
 install: install-cdc-decoders
 clean: clean-cdc-decoders
--- a/src/backend/distributed/cdc/Makefile.decoder
+++ b/src/backend/distributed/cdc/Makefile.decoder
@ -1,19 +0,0 @@
 MODULE_big = citus_$(DECODER)
 citus_decoders_dir = $(DESTDIR)$(pkglibdir)/citus_decoders
 citus_top_builddir = ../../../../..
 citus_subdir = src/backend/distributed/cdc/cdc_$(DECODER)
 OBJS += cdc_decoder.o cdc_decoder_utils.o
 include $(citus_top_builddir)/Makefile.global
 override CFLAGS += -DDECODER=\"$(DECODER)\" -I$(citus_abs_top_srcdir)/include
 override CPPFLAGS += -DDECODER=\"$(DECODER)\" -I$(citus_abs_top_srcdir)/include
 install: install-cdc
 install-cdc:
 	mkdir -p '$(citus_decoders_dir)'
 	$(INSTALL_SHLIB) citus_$(DECODER)$(DLSUFFIX) '$(citus_decoders_dir)/$(DECODER)$(DLSUFFIX)'
--- a/src/backend/distributed/cdc/cdc_decoder.c
+++ b/src/backend/distributed/cdc/cdc_decoder.c
@ -1,573 +0,0 @@
 /*-------------------------------------------------------------------------
 *
 * cdc_decoder.c
 *		CDC Decoder plugin for Citus
 *
 * Copyright (c) Citus Data, Inc.
 *
 *-------------------------------------------------------------------------
 */
 #include "postgres.h"
 #include "cdc_decoder_utils.h"
 #include "fmgr.h"
 #include "access/genam.h"
 #include "catalog/pg_namespace.h"
 #include "catalog/pg_publication.h"
 #include "commands/extension.h"
 #include "common/hashfn.h"
 #include "utils/lsyscache.h"
 #include "utils/rel.h"
 #include "utils/typcache.h"
 #include "pg_version_constants.h"
 PG_MODULE_MAGIC;
 extern void _PG_output_plugin_init(OutputPluginCallbacks *cb);
 static LogicalDecodeChangeCB ouputPluginChangeCB;
 static void InitShardToDistributedTableMap(void);
 static void PublishDistributedTableChanges(LogicalDecodingContext *ctx,
 										   ReorderBufferTXN *txn,
 										   Relation relation,
 										   ReorderBufferChange *change);
 static bool replication_origin_filter_cb(LogicalDecodingContext *ctx, RepOriginId
 										 origin_id);
 static void TranslateChangesIfSchemaChanged(Relation relation, Relation targetRelation,
 											ReorderBufferChange *change);
 static void TranslateAndPublishRelationForCDC(LogicalDecodingContext *ctx,
 											  ReorderBufferTXN *txn,
 											  Relation relation,
 											  ReorderBufferChange *change, Oid shardId,
 											  Oid targetRelationid);
 typedef struct
 {
 	uint64 shardId;
 	Oid distributedTableId;
 	bool isReferenceTable;
 	bool isNull;
 } ShardIdHashEntry;
 static HTAB *shardToDistributedTableMap = NULL;
 static void cdc_change_cb(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
 						  Relation relation, ReorderBufferChange *change);
 /* build time macro for base decoder plugin name for CDC and Shard Split. */
 #ifndef DECODER
 #define DECODER "pgoutput"
 #endif
 #define DECODER_INIT_FUNCTION_NAME "_PG_output_plugin_init"
 #define CITUS_SHARD_TRANSFER_SLOT_PREFIX "citus_shard_"
 #define CITUS_SHARD_TRANSFER_SLOT_PREFIX_SIZE (sizeof(CITUS_SHARD_TRANSFER_SLOT_PREFIX) - \
 											   1)
 /*
 * Postgres uses 'pgoutput' as default plugin for logical replication.
 * We want to reuse Postgres pgoutput's functionality as much as possible.
 * Hence we load all the functions of this plugin and override as required.
 */
 void
 _PG_output_plugin_init(OutputPluginCallbacks *cb)
 {
 	elog(LOG, "Initializing CDC decoder");
 	/*
 	 * We build custom .so files whose name matches common decoders (pgoutput, wal2json)
 	 * and place them in $libdir/citus_decoders/ such that administrators can configure
 	 * dynamic_library_path to include this directory, and users can then use the
 	 * regular decoder names when creating replications slots.
 	 *
 	 * To load the original decoder, we need to remove citus_decoders/ from the
 	 * dynamic_library_path.
 	 */
 	char *originalDLP = Dynamic_library_path;
 	Dynamic_library_path = RemoveCitusDecodersFromPaths(Dynamic_library_path);
 	LogicalOutputPluginInit plugin_init =
 		(LogicalOutputPluginInit) (void *)
 		load_external_function(DECODER,
 							   DECODER_INIT_FUNCTION_NAME,
 							   false, NULL);
 	if (plugin_init == NULL)
 	{
 		elog(ERROR, "output plugins have to declare the _PG_output_plugin_init symbol");
 	}
 	/* in case this session is used for different replication slots */
 	Dynamic_library_path = originalDLP;
 	/* ask the output plugin to fill the callback struct */
 	plugin_init(cb);
 	/* Initialize the Shard Id to Distributed Table id mapping hash table.*/
 	InitShardToDistributedTableMap();
 	/* actual pgoutput callback function will be called  */
 	ouputPluginChangeCB = cb->change_cb;
 	cb->change_cb = cdc_change_cb;
 	cb->filter_by_origin_cb = replication_origin_filter_cb;
 }
 /*
 *  Check if the replication slot is for Shard transfer by checking for prefix.
 */
 inline static
 bool
 IsShardTransferSlot(char *replicationSlotName)
 {
 	return strncmp(replicationSlotName, CITUS_SHARD_TRANSFER_SLOT_PREFIX,
 				   CITUS_SHARD_TRANSFER_SLOT_PREFIX_SIZE) == 0;
 }
 /*
 * shard_split_and_cdc_change_cb function emits the incoming tuple change
 * to the appropriate destination shard.
 */
 static void
 cdc_change_cb(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
 			  Relation relation, ReorderBufferChange *change)
 {
 	/*
 	 * If Citus has not been loaded yet, pass the changes
 	 * through to the undrelying decoder plugin.
 	 */
 	if (!CdcCitusHasBeenLoaded())
 	{
 		ouputPluginChangeCB(ctx, txn, relation, change);
 		return;
 	}
 	/* check if the relation is publishable.*/
 	if (!is_publishable_relation(relation))
 	{
 		return;
 	}
 	char *replicationSlotName = ctx->slot->data.name.data;
 	if (replicationSlotName == NULL)
 	{
 		elog(ERROR, "Replication slot name is NULL!");
 		return;
 	}
 	/* If the slot is for internal shard operations, call the base plugin's call back. */
 	if (IsShardTransferSlot(replicationSlotName))
 	{
 		ouputPluginChangeCB(ctx, txn, relation, change);
 		return;
 	}
 	/* Transalate the changes from shard to distributes table and publish. */
 	PublishDistributedTableChanges(ctx, txn, relation, change);
 }
 /*
 * InitShardToDistributedTableMap initializes the hash table that is used to
 * translate the changes in the shard table to the changes in the distributed table.
 */
 static void
 InitShardToDistributedTableMap()
 {
 	HASHCTL info;
 	memset(&info, 0, sizeof(info));
 	info.keysize = sizeof(uint64);
 	info.entrysize = sizeof(ShardIdHashEntry);
 	info.hash = tag_hash;
 	info.hcxt = CurrentMemoryContext;
 	int hashFlags = (HASH_ELEM | HASH_CONTEXT | HASH_FUNCTION);
 	shardToDistributedTableMap = hash_create("CDC Decoder translation hash table", 1024,
 											 &info, hashFlags);
 }
 /*
 * AddShardIdToHashTable adds the shardId to the hash table.
 */
 static Oid
 AddShardIdToHashTable(uint64 shardId, ShardIdHashEntry *entry)
 {
 	entry->shardId = shardId;
 	entry->distributedTableId = CdcLookupShardRelationFromCatalog(shardId, true);
 	entry->isReferenceTable = CdcIsReferenceTableViaCatalog(entry->distributedTableId);
 	return entry->distributedTableId;
 }
 static Oid
 LookupDistributedTableIdForShardId(uint64 shardId, bool *isReferenceTable)
 {
 	bool found;
 	Oid distributedTableId = InvalidOid;
 	ShardIdHashEntry *entry = (ShardIdHashEntry *) hash_search(shardToDistributedTableMap,
 															   &shardId,
 															   HASH_ENTER,
 															   &found);
 	if (found)
 	{
 		distributedTableId = entry->distributedTableId;
 	}
 	else
 	{
 		distributedTableId = AddShardIdToHashTable(shardId, entry);
 	}
 	*isReferenceTable = entry->isReferenceTable;
 	return distributedTableId;
 }
 /*
 * replication_origin_filter_cb call back function filters out publication of changes
 * originated from any other node other than the current node. This is
 * identified by the "origin_id" of the changes. The origin_id is set to
 * a non-zero value in the origin node as part of WAL replication for internal
 * operations like shard split/moves/create_distributed_table etc.
 */
 static bool
 replication_origin_filter_cb(LogicalDecodingContext *ctx, RepOriginId origin_id)
 {
 	return  (origin_id != InvalidRepOriginId);
 }
 /*
 * This function is responsible for translating the changes in the shard table to
 * the changes in the shell table and publishing the changes as a change to the
 * distributed table so that CDD clients are not aware of the shard tables. It also
 * handles schema changes to the distributed table.
 */
 static void
 TranslateAndPublishRelationForCDC(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
 								  Relation relation, ReorderBufferChange *change, Oid
 								  shardId, Oid targetRelationid)
 {
 	/* Get the distributed table's relation for this shard.*/
 	Relation targetRelation = RelationIdGetRelation(targetRelationid);
 	/*
 	 * Check if there has been a schema change (such as a dropped column), by comparing
 	 * the number of attributes in the shard table and the shell table.
 	 */
 	TranslateChangesIfSchemaChanged(relation, targetRelation, change);
 	/*
 	 * Publish the change to the shard table as the change in the distributed table,
 	 * so that the CDC client can see the change in the distributed table,
 	 * instead of the shard table, by calling the pgoutput's callback function.
 	 */
 	ouputPluginChangeCB(ctx, txn, targetRelation, change);
 	RelationClose(targetRelation);
 }
 /*
 * PublishChangesIfCdcSlot checks if the current slot is a CDC slot. If so, it publishes
 * the changes as the change for the distributed table instead of shard.
 * If not, it returns false. It also skips the Citus metadata tables.
 */
 static void
 PublishDistributedTableChanges(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
 							   Relation relation, ReorderBufferChange *change)
 {
 	char *shardRelationName = RelationGetRelationName(relation);
 	/* Skip publishing CDC changes for any system relations in pg_catalog*/
 	if (relation->rd_rel->relnamespace == PG_CATALOG_NAMESPACE)
 	{
 		return;
 	}
 	/* Check if the relation is a distributed table by checking for shard name.	*/
 	uint64 shardId = CdcExtractShardIdFromTableName(shardRelationName, true);
 	/* If this relation is not distributed, call the pgoutput's callback and return. */
 	if (shardId == INVALID_SHARD_ID)
 	{
 		ouputPluginChangeCB(ctx, txn, relation, change);
 		return;
 	}
 	bool isReferenceTable = false;
 	Oid distRelationId = LookupDistributedTableIdForShardId(shardId, &isReferenceTable);
 	if (distRelationId == InvalidOid)
 	{
 		ouputPluginChangeCB(ctx, txn, relation, change);
 		return;
 	}
 	/* Publish changes for reference table only from the coordinator node. */
 	if (isReferenceTable && !CdcIsCoordinator())
 	{
 		return;
 	}
 	/* translate and publish from shard relation to distributed table relation for CDC. */
 	TranslateAndPublishRelationForCDC(ctx, txn, relation, change, shardId,
 									  distRelationId);
 }
 /*
 * GetTupleForTargetSchemaForCdc returns a heap tuple with the data from sourceRelationTuple
 * to match the schema in targetRelDesc. Either or both source and target relations may have
 * dropped columns. This function handles it by adding NULL values for dropped columns in
 * target relation and skipping dropped columns in source relation. It returns a heap tuple
 * adjusted to the current schema of the target relation.
 */
 static HeapTuple
 GetTupleForTargetSchemaForCdc(HeapTuple sourceRelationTuple,
 							  TupleDesc sourceRelDesc,
 							  TupleDesc targetRelDesc)
 {
 	/* Allocate memory for sourceValues and sourceNulls arrays. */
 	Datum *sourceValues = (Datum *) palloc0(sourceRelDesc->natts * sizeof(Datum));
 	bool *sourceNulls = (bool *) palloc0(sourceRelDesc->natts * sizeof(bool));
 	/* Deform the source tuple to sourceValues and sourceNulls arrays. */
 	heap_deform_tuple(sourceRelationTuple, sourceRelDesc, sourceValues,
 					  sourceNulls);
 	/* This is the next field to Read in the source relation */
 	uint32 sourceIndex = 0;
 	uint32 targetIndex = 0;
 	/* Allocate memory for sourceValues and sourceNulls arrays. */
 	Datum *targetValues = (Datum *) palloc0(targetRelDesc->natts * sizeof(Datum));
 	bool *targetNulls = (bool *) palloc0(targetRelDesc->natts * sizeof(bool));
 	/* Loop through all source and target attributes one by one and handle any dropped attributes.*/
 	while (targetIndex < targetRelDesc->natts)
 	{
 		/* If this target attribute has been dropped, add a NULL attribute in targetValues and continue.*/
 		if (TupleDescAttr(targetRelDesc, targetIndex)->attisdropped)
 		{
 			Datum nullDatum = (Datum) 0;
 			targetValues[targetIndex] = nullDatum;
 			targetNulls[targetIndex] = true;
 			targetIndex++;
 		}
 		/* If this source attribute has been dropped, just skip this source attribute.*/
 		else if (TupleDescAttr(sourceRelDesc, sourceIndex)->attisdropped)
 		{
 			sourceIndex++;
 			continue;
 		}
 		/* If both source and target attributes are not dropped, add the attribute field to targetValues. */
 		else if (sourceIndex < sourceRelDesc->natts)
 		{
 			targetValues[targetIndex] = sourceValues[sourceIndex];
 			targetNulls[targetIndex] = sourceNulls[sourceIndex];
 			sourceIndex++;
 			targetIndex++;
 		}
 		else
 		{
 			/* If there are no more source fields, add a NULL field in targetValues. */
 			Datum nullDatum = (Datum) 0;
 			targetValues[targetIndex] = nullDatum;
 			targetNulls[targetIndex] = true;
 			targetIndex++;
 		}
 	}
 	/* Form a new tuple from the target values created by the above loop. */
 	HeapTuple targetRelationTuple = heap_form_tuple(targetRelDesc, targetValues,
 													targetNulls);
 	return targetRelationTuple;
 }
 /* HasSchemaChanged function returns if there any schema changes between source and target relations.*/
 static bool
 HasSchemaChanged(TupleDesc sourceRelationDesc, TupleDesc targetRelationDesc)
 {
 	bool hasSchemaChanged = (sourceRelationDesc->natts != targetRelationDesc->natts);
 	if (hasSchemaChanged)
 	{
 		return true;
 	}
 	for (uint32 i = 0; i < sourceRelationDesc->natts; i++)
 	{
 		if (TupleDescAttr(sourceRelationDesc, i)->attisdropped ||
 			TupleDescAttr(targetRelationDesc, i)->attisdropped)
 		{
 			hasSchemaChanged = true;
 			break;
 		}
 	}
 	return hasSchemaChanged;
 }
 /*
 * TranslateChangesIfSchemaChanged translates the tuples ReorderBufferChange
 * if there is a schema change between source and target relations.
 */
 static void
 TranslateChangesIfSchemaChanged(Relation sourceRelation, Relation targetRelation,
 								ReorderBufferChange *change)
 {
 	TupleDesc sourceRelationDesc = RelationGetDescr(sourceRelation);
 	TupleDesc targetRelationDesc = RelationGetDescr(targetRelation);
 	/* if there are no changes between source and target relations, return. */
 	if (!HasSchemaChanged(sourceRelationDesc, targetRelationDesc))
 	{
 		return;
 	}
 #if PG_VERSION_NUM >= PG_VERSION_17
 	/* Check the ReorderBufferChange's action type and handle them accordingly.*/
 	switch (change->action)
 	{
 		case REORDER_BUFFER_CHANGE_INSERT:
 		{
 			/* For insert action, only new tuple should always be translated*/
 			HeapTuple sourceRelationNewTuple = change->data.tp.newtuple;
 			HeapTuple targetRelationNewTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationNewTuple, sourceRelationDesc, targetRelationDesc);
 			change->data.tp.newtuple = targetRelationNewTuple;
 			break;
 		}
 		/*
 		 * For update changes both old and new tuples need to be translated for target relation
 		 * if the REPLICA IDENTITY is set to FULL. Otherwise, only the new tuple needs to be
 		 * translated for target relation.
 		 */
 		case REORDER_BUFFER_CHANGE_UPDATE:
 		{
 			/* For update action, new tuple should always be translated*/
 			/* Get the new tuple from the ReorderBufferChange, and translate it to target relation. */
 			HeapTuple sourceRelationNewTuple = change->data.tp.newtuple;
 			HeapTuple targetRelationNewTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationNewTuple, sourceRelationDesc, targetRelationDesc);
 			change->data.tp.newtuple = targetRelationNewTuple;
 			/*
 			 * Format oldtuple according to the target relation. If the column values of replica
 			 * identiy change, then the old tuple is non-null and needs to be formatted according
 			 * to the target relation schema.
 			 */
 			if (change->data.tp.oldtuple != NULL)
 			{
 				HeapTuple sourceRelationOldTuple = change->data.tp.oldtuple;
 				HeapTuple targetRelationOldTuple = GetTupleForTargetSchemaForCdc(
 					sourceRelationOldTuple,
 					sourceRelationDesc,
 					targetRelationDesc);
 				change->data.tp.oldtuple = targetRelationOldTuple;
 			}
 			break;
 		}
 		case REORDER_BUFFER_CHANGE_DELETE:
 		{
 			/* For delete action, only old tuple should be translated*/
 			HeapTuple sourceRelationOldTuple = change->data.tp.oldtuple;
 			HeapTuple targetRelationOldTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationOldTuple,
 				sourceRelationDesc,
 				targetRelationDesc);
 			change->data.tp.oldtuple = targetRelationOldTuple;
 			break;
 		}
 		default:
 		{
 			/* Do nothing for other action types. */
 			break;
 		}
 	}
 #else
 	/* Check the ReorderBufferChange's action type and handle them accordingly.*/
 	switch (change->action)
 	{
 		case REORDER_BUFFER_CHANGE_INSERT:
 		{
 			/* For insert action, only new tuple should always be translated*/
 			HeapTuple sourceRelationNewTuple = &(change->data.tp.newtuple->tuple);
 			HeapTuple targetRelationNewTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationNewTuple, sourceRelationDesc, targetRelationDesc);
 			change->data.tp.newtuple->tuple = *targetRelationNewTuple;
 			break;
 		}
 		/*
 		 * For update changes both old and new tuples need to be translated for target relation
 		 * if the REPLICA IDENTITY is set to FULL. Otherwise, only the new tuple needs to be
 		 * translated for target relation.
 		 */
 		case REORDER_BUFFER_CHANGE_UPDATE:
 		{
 			/* For update action, new tuple should always be translated*/
 			/* Get the new tuple from the ReorderBufferChange, and translate it to target relation. */
 			HeapTuple sourceRelationNewTuple = &(change->data.tp.newtuple->tuple);
 			HeapTuple targetRelationNewTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationNewTuple, sourceRelationDesc, targetRelationDesc);
 			change->data.tp.newtuple->tuple = *targetRelationNewTuple;
 			/*
 			 * Format oldtuple according to the target relation. If the column values of replica
 			 * identiy change, then the old tuple is non-null and needs to be formatted according
 			 * to the target relation schema.
 			 */
 			if (change->data.tp.oldtuple != NULL)
 			{
 				HeapTuple sourceRelationOldTuple = &(change->data.tp.oldtuple->tuple);
 				HeapTuple targetRelationOldTuple = GetTupleForTargetSchemaForCdc(
 					sourceRelationOldTuple,
 					sourceRelationDesc,
 					targetRelationDesc);
 				change->data.tp.oldtuple->tuple = *targetRelationOldTuple;
 			}
 			break;
 		}
 		case REORDER_BUFFER_CHANGE_DELETE:
 		{
 			/* For delete action, only old tuple should be translated*/
 			HeapTuple sourceRelationOldTuple = &(change->data.tp.oldtuple->tuple);
 			HeapTuple targetRelationOldTuple = GetTupleForTargetSchemaForCdc(
 				sourceRelationOldTuple,
 				sourceRelationDesc,
 				targetRelationDesc);
 			change->data.tp.oldtuple->tuple = *targetRelationOldTuple;
 			break;
 		}
 		default:
 		{
 			/* Do nothing for other action types. */
 			break;
 		}
 	}
 #endif
 }
--- a/src/backend/distributed/cdc/cdc_decoder_utils.c
+++ b/src/backend/distributed/cdc/cdc_decoder_utils.c
@ -1,447 +0,0 @@
 /*-------------------------------------------------------------------------
 *
 * cdc_decoder_utils.c
 *		CDC Decoder plugin utility functions for Citus
 *
 * Copyright (c) Citus Data, Inc.
 *
 *-------------------------------------------------------------------------
 */
 #include "postgres.h"
 #include "cdc_decoder_utils.h"
 #include "fmgr.h"
 #include "miscadmin.h"
 #include "access/genam.h"
 #include "access/heapam.h"
 #include "catalog/pg_namespace.h"
 #include "commands/extension.h"
 #include "common/hashfn.h"
 #include "common/string.h"
 #include "utils/fmgroids.h"
 #include "utils/lsyscache.h"
 #include "utils/typcache.h"
 #include "distributed/pg_dist_partition.h"
 #include "distributed/pg_dist_shard.h"
 #include "distributed/relay_utility.h"
 static int32 LocalGroupId = -1;
 static Oid PgDistLocalGroupRelationId = InvalidOid;
 static Oid PgDistShardRelationId = InvalidOid;
 static Oid PgDistShardShardidIndexId = InvalidOid;
 static Oid PgDistPartitionRelationId = InvalidOid;
 static Oid PgDistPartitionLogicalrelidIndexId = InvalidOid;
 static bool IsCitusExtensionLoaded = false;
 #define COORDINATOR_GROUP_ID 0
 #define InvalidRepOriginId 0
 #define Anum_pg_dist_local_groupid 1
 #define GROUP_ID_UPGRADING -2
 static Oid DistLocalGroupIdRelationId(void);
 static int32 CdcGetLocalGroupId(void);
 static HeapTuple CdcPgDistPartitionTupleViaCatalog(Oid relationId);
 /*
 * DistLocalGroupIdRelationId returns the relation id of the pg_dist_local_group
 */
 static Oid
 DistLocalGroupIdRelationId(void)
 {
 	if (PgDistLocalGroupRelationId == InvalidOid)
 	{
 		PgDistLocalGroupRelationId = get_relname_relid("pg_dist_local_group",
 													   PG_CATALOG_NAMESPACE);
 	}
 	return PgDistLocalGroupRelationId;
 }
 /*
 * DistShardRelationId returns the relation id of the pg_dist_shard
 */
 static Oid
 DistShardRelationId(void)
 {
 	if (PgDistShardRelationId == InvalidOid)
 	{
 		PgDistShardRelationId = get_relname_relid("pg_dist_shard", PG_CATALOG_NAMESPACE);
 	}
 	return PgDistShardRelationId;
 }
 /*
 * DistShardShardidIndexId returns the relation id of the pg_dist_shard_shardid_index
 */
 static Oid
 DistShardShardidIndexId(void)
 {
 	if (PgDistShardShardidIndexId == InvalidOid)
 	{
 		PgDistShardShardidIndexId = get_relname_relid("pg_dist_shard_shardid_index",
 													  PG_CATALOG_NAMESPACE);
 	}
 	return PgDistShardShardidIndexId;
 }
 /*
 * DistPartitionRelationId returns the relation id of the pg_dist_partition
 */
 static Oid
 DistPartitionRelationId(void)
 {
 	if (PgDistPartitionRelationId == InvalidOid)
 	{
 		PgDistPartitionRelationId = get_relname_relid("pg_dist_partition",
 													  PG_CATALOG_NAMESPACE);
 	}
 	return PgDistPartitionRelationId;
 }
 static Oid
 DistPartitionLogicalRelidIndexId(void)
 {
 	if (PgDistPartitionLogicalrelidIndexId == InvalidOid)
 	{
 		PgDistPartitionLogicalrelidIndexId = get_relname_relid(
 			"pg_dist_partition_logicalrelid_index", PG_CATALOG_NAMESPACE);
 	}
 	return PgDistPartitionLogicalrelidIndexId;
 }
 /*
 * CdcIsCoordinator function returns true if this node is identified as the
 * schema/coordinator/master node of the cluster.
 */
 bool
 CdcIsCoordinator(void)
 {
 	return (CdcGetLocalGroupId() == COORDINATOR_GROUP_ID);
 }
 /*
 * CdcCitusHasBeenLoaded function returns true if the citus extension has been loaded.
 */
 bool
 CdcCitusHasBeenLoaded()
 {
 	if (!IsCitusExtensionLoaded)
 	{
 		IsCitusExtensionLoaded = (get_extension_oid("citus", true) != InvalidOid);
 	}
 	return IsCitusExtensionLoaded;
 }
 /*
 * ExtractShardIdFromTableName tries to extract shard id from the given table name,
 * and returns the shard id if table name is formatted as shard name.
 * Else, the function returns INVALID_SHARD_ID.
 */
 uint64
 CdcExtractShardIdFromTableName(const char *tableName, bool missingOk)
 {
 	char *shardIdStringEnd = NULL;
 	/* find the last underscore and increment for shardId string */
 	char *shardIdString = strrchr(tableName, SHARD_NAME_SEPARATOR);
 	if (shardIdString == NULL && !missingOk)
 	{
 		ereport(ERROR, (errmsg("could not extract shardId from table name \"%s\"",
 							   tableName)));
 	}
 	else if (shardIdString == NULL && missingOk)
 	{
 		return INVALID_SHARD_ID;
 	}
 	shardIdString++;
 	errno = 0;
 	uint64 shardId = strtoull(shardIdString, &shardIdStringEnd, 0);
 	if (errno != 0 || (*shardIdStringEnd != '\0'))
 	{
 		if (!missingOk)
 		{
 			ereport(ERROR, (errmsg("could not extract shardId from table name \"%s\"",
 								   tableName)));
 		}
 		else
 		{
 			return INVALID_SHARD_ID;
 		}
 	}
 	return shardId;
 }
 /*
 * CdcGetLocalGroupId returns the group identifier of the local node. The
 * function assumes that pg_dist_local_group has exactly one row and has at
 * least one column. Otherwise, the function errors out.
 */
 static int32
 CdcGetLocalGroupId(void)
 {
 	ScanKeyData scanKey[1];
 	int scanKeyCount = 0;
 	int32 groupId = 0;
 	/*
 	 * Already set the group id, no need to read the heap again.
 	 */
 	if (LocalGroupId != -1)
 	{
 		return LocalGroupId;
 	}
 	Oid localGroupTableOid = DistLocalGroupIdRelationId();
 	if (localGroupTableOid == InvalidOid)
 	{
 		return 0;
 	}
 	Relation pgDistLocalGroupId = table_open(localGroupTableOid, AccessShareLock);
 	SysScanDesc scanDescriptor = systable_beginscan(pgDistLocalGroupId,
 													InvalidOid, false,
 													NULL, scanKeyCount, scanKey);
 	TupleDesc tupleDescriptor = RelationGetDescr(pgDistLocalGroupId);
 	HeapTuple heapTuple = systable_getnext(scanDescriptor);
 	if (HeapTupleIsValid(heapTuple))
 	{
 		bool isNull = false;
 		Datum groupIdDatum = heap_getattr(heapTuple,
 										  Anum_pg_dist_local_groupid,
 										  tupleDescriptor, &isNull);
 		groupId = DatumGetInt32(groupIdDatum);
 		/* set the local cache variable */
 		LocalGroupId = groupId;
 	}
 	else
 	{
 		/*
 		 * Upgrade is happening. When upgrading postgres, pg_dist_local_group is
 		 * temporarily empty before citus_finish_pg_upgrade() finishes execution.
 		 */
 		groupId = GROUP_ID_UPGRADING;
 	}
 	systable_endscan(scanDescriptor);
 	table_close(pgDistLocalGroupId, AccessShareLock);
 	return groupId;
 }
 /*
 * CdcLookupShardRelationFromCatalog returns the logical relation oid a shard belongs to.
 *
 * Errors out if the shardId does not exist and missingOk is false.
 * Returns InvalidOid if the shardId does not exist and missingOk is true.
 */
 Oid
 CdcLookupShardRelationFromCatalog(int64 shardId, bool missingOk)
 {
 	ScanKeyData scanKey[1];
 	int scanKeyCount = 1;
 	Form_pg_dist_shard shardForm = NULL;
 	Relation pgDistShard = table_open(DistShardRelationId(), AccessShareLock);
 	Oid relationId = InvalidOid;
 	ScanKeyInit(&scanKey[0], Anum_pg_dist_shard_shardid,
 				BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum(shardId));
 	SysScanDesc scanDescriptor = systable_beginscan(pgDistShard,
 													DistShardShardidIndexId(), true,
 													NULL, scanKeyCount, scanKey);
 	HeapTuple heapTuple = systable_getnext(scanDescriptor);
 	if (!HeapTupleIsValid(heapTuple) && !missingOk)
 	{
 		ereport(ERROR, (errmsg("could not find valid entry for shard "
 							   UINT64_FORMAT, shardId)));
 	}
 	if (!HeapTupleIsValid(heapTuple))
 	{
 		relationId = InvalidOid;
 	}
 	else
 	{
 		shardForm = (Form_pg_dist_shard) GETSTRUCT(heapTuple);
 		relationId = shardForm->logicalrelid;
 	}
 	systable_endscan(scanDescriptor);
 	table_close(pgDistShard, NoLock);
 	return relationId;
 }
 /*
 * CdcPgDistPartitionTupleViaCatalog is a helper function that searches
 * pg_dist_partition for the given relationId. The caller is responsible
 * for ensuring that the returned heap tuple is valid before accessing
 * its fields.
 */
 static HeapTuple
 CdcPgDistPartitionTupleViaCatalog(Oid relationId)
 {
 	const int scanKeyCount = 1;
 	ScanKeyData scanKey[1];
 	bool indexOK = true;
 	Relation pgDistPartition = table_open(DistPartitionRelationId(), AccessShareLock);
 	ScanKeyInit(&scanKey[0], Anum_pg_dist_partition_logicalrelid,
 				BTEqualStrategyNumber, F_OIDEQ, ObjectIdGetDatum(relationId));
 	SysScanDesc scanDescriptor = systable_beginscan(pgDistPartition,
 													DistPartitionLogicalRelidIndexId(),
 													indexOK, NULL, scanKeyCount, scanKey);
 	HeapTuple partitionTuple = systable_getnext(scanDescriptor);
 	if (HeapTupleIsValid(partitionTuple))
 	{
 		/* callers should have the tuple in their memory contexts */
 		partitionTuple = heap_copytuple(partitionTuple);
 	}
 	systable_endscan(scanDescriptor);
 	table_close(pgDistPartition, AccessShareLock);
 	return partitionTuple;
 }
 /*
 * CdcIsReferenceTableViaCatalog gets a relationId and returns true if the relation
 * is a reference table and false otherwise.
 */
 char
 CdcIsReferenceTableViaCatalog(Oid relationId)
 {
 	HeapTuple partitionTuple = CdcPgDistPartitionTupleViaCatalog(relationId);
 	if (!HeapTupleIsValid(partitionTuple))
 	{
 		return false;
 	}
 	Datum datumArray[Natts_pg_dist_partition];
 	bool isNullArray[Natts_pg_dist_partition];
 	Relation pgDistPartition = table_open(DistPartitionRelationId(), AccessShareLock);
 	TupleDesc tupleDescriptor = RelationGetDescr(pgDistPartition);
 	heap_deform_tuple(partitionTuple, tupleDescriptor, datumArray, isNullArray);
 	if (isNullArray[Anum_pg_dist_partition_partmethod - 1] ||
 		isNullArray[Anum_pg_dist_partition_repmodel - 1])
 	{
 		/*
 		 * partition method and replication model cannot be NULL,
 		 * still let's make sure
 		 */
 		heap_freetuple(partitionTuple);
 		table_close(pgDistPartition, NoLock);
 		return false;
 	}
 	Datum partitionMethodDatum = datumArray[Anum_pg_dist_partition_partmethod - 1];
 	char partitionMethodChar = DatumGetChar(partitionMethodDatum);
 	Datum replicationModelDatum = datumArray[Anum_pg_dist_partition_repmodel - 1];
 	char replicationModelChar = DatumGetChar(replicationModelDatum);
 	heap_freetuple(partitionTuple);
 	table_close(pgDistPartition, NoLock);
 	/*
 	 * A table is a reference table when its partition method is 'none'
 	 * and replication model is 'two phase commit'
 	 */
 	return partitionMethodChar == DISTRIBUTE_BY_NONE &&
 		   replicationModelChar == REPLICATION_MODEL_2PC;
 }
 /*
 * RemoveCitusDecodersFromPaths removes a path ending in citus_decoders
 * from the given input paths.
 */
 char *
 RemoveCitusDecodersFromPaths(char *paths)
 {
 	if (strlen(paths) == 0)
 	{
 		/* dynamic_library_path is empty */
 		return paths;
 	}
 	StringInfo newPaths = makeStringInfo();
 	char *remainingPaths = paths;
 	for (;;)
 	{
 		int pathLength = 0;
 		char *pathStart = first_path_var_separator(remainingPaths);
 		if (pathStart == remainingPaths)
 		{
 			/*
 			 * This will error out in find_in_dynamic_libpath, return
 			 * original value here.
 			 */
 			return paths;
 		}
 		else if (pathStart == NULL)
 		{
 			/* final path */
 			pathLength = strlen(remainingPaths);
 		}
 		else
 		{
 			/* more paths remaining */
 			pathLength = pathStart - remainingPaths;
 		}
 		char *currentPath = palloc(pathLength + 1);
 		strlcpy(currentPath, remainingPaths, pathLength + 1);
 		canonicalize_path(currentPath);
 		if (!pg_str_endswith(currentPath, "/citus_decoders"))
 		{
 			appendStringInfo(newPaths, "%s%s", newPaths->len > 0 ? ":" : "", currentPath);
 		}
 		if (remainingPaths[pathLength] == '\0')
 		{
 			/* end of string */
 			break;
 		}
 		remainingPaths += pathLength + 1;
 	}
 	return newPaths->data;
 }
--- a/src/backend/distributed/cdc/cdc_decoder_utils.h
+++ b/src/backend/distributed/cdc/cdc_decoder_utils.h
@ -1,36 +0,0 @@
 /*-------------------------------------------------------------------------
 *
 * cdc_decoder_utils.h
 *	  Utility functions and declerations for cdc decoder.
 *
 * Copyright (c) Citus Data, Inc.
 *
 *-------------------------------------------------------------------------
 */
 #ifndef CITUS_CDC_DECODER_H
 #define CITUS_CDC_DECODER_H
 #include "postgres.h"
 #include "c.h"
 #include "fmgr.h"
 #include "replication/logical.h"
 #define InvalidRepOriginId 0
 #define INVALID_SHARD_ID 0
 bool CdcIsCoordinator(void);
 uint64 CdcExtractShardIdFromTableName(const char *tableName, bool missingOk);
 Oid CdcLookupShardRelationFromCatalog(int64 shardId, bool missingOk);
 char CdcIsReferenceTableViaCatalog(Oid relationId);
 bool CdcCitusHasBeenLoaded(void);
 char * RemoveCitusDecodersFromPaths(char *paths);
 #endif   /* CITUS_CDC_DECODER_UTILS_H */
--- a/src/backend/distributed/citus.control
+++ b/src/backend/distributed/citus.control
@ -1,6 +1,6 @@
 # Citus extension
 comment = 'Citus distributed database'
-default_version = '13.2-1'
+default_version = '11.1-1'
 module_pathname = '$libdir/citus'
 relocatable = false
 schema = pg_catalog
--- a/src/backend/distributed/clock/README.md
+++ b/src/backend/distributed/clock/README.md
@ -1,244 +0,0 @@
 # Cluster Clock
 ### Requirement:
 Many distributed applications need to track the changes in the same order as they are applied on the database. The changes can be to databases or objects within them, either within a single node or across the sharded cluster.
 ### Definitions
 **Total ordering** - Every pair of change events can be placed in some order.
 **Causal ordering** - Only events that are causally related (an event A caused an event B) can be ordered i.e., it's only a partial order - sometimes events happen independently with no possible causal relationship, such events are treated to concurrent.
 **Sequential consistency** - All writes must be seen in the same order by all processes.
 **Causal consistency** - Causally related writes must be seen in the same order.
 Transactions on a single node system naturally provide a total and sequential ordering guarantees for client read and write operations as all operations are routed to the same node, but there are challenges for a multi node distributed system, such as, Citus.
 One possible way to totally order all the changes in the system is to timestamp all the events with a global physical clock or a centralized logical clock. Thus, observing the events in the increasing order of the timestamp will give the total ordering of events. For both the performance and cost reasons such solutions are impractical. In the absence of total ordering, a little weaker ordering is the **causal order**.
 Causal order is defined as a model that preserves a partial order of events in a distributed system. If an event
 1.  A causes another event B, every other process in the system observes the event A before observing event B.
 2.  Causal order is transitive: if A causes B, and B causes C, then A causes C.
 3.  Non causally ordered events are treated as concurrent.
 Causal consistency is a weak form of consistency that preserves the order of causally related operations. The causal consistency model can be refined into four session guarantees.
 1.  Read Your Writes: If a process performs a write, the same process later observes the result of its write.
 6.  Monotonic Reads: The set of writes observed (read) by a process is guaranteed to be monotonically increasing.
 7.  Writes Follow Reads: If some process performs a read followed by a write, and another process observes the result of the write, then it can also observe the read.
 8.  Monotonic Writes: If some process performs a write, followed sometime later by another write, other processes will observe them in the same order.
 ### Hybrid Logical Clock (HLC)
 HLC provides a way to get the causality relationship like logical clocks. It can also be used for backup/recovery too as the logical clock value is maintained close to the wall clock time. HLC consists of
 LC - Logical clock
 C - Counter
 Clock components - Unsigned 64 bit <LC, C>
 Epoch Milliseconds ( LC ) | Logical counter ( C )|
 |--|--|
 | 42 bits | 22 bits |
 2^42 milliseconds - 4398046511104 milliseconds, which is ~139 years.
 2^22 ticks - maximum of four million operations per millisecond.
 ### UDFs
 A new UDF `citus_get_cluster_clock`() that returns a monotonically increasing logical clock. Clock guarantees to never go back in value after restarts and makes best attempt to keep the value close to UNIX epoch time in milliseconds.
 A new UDF `citus_get_transaction_clock`(), when called by the user, returns the logical causal clock timestamp current transaction,
 Internally, this is the maximum clock among all transaction nodes, and
 all nodes move to the new clock.
 ### GUC
 A new GUC parameter, "**citus.enable_cluster_clock**", If clocks go bad for any reason, this serves as a safety valve to avoid the need to change the application and (re)deploy it.
 ### Sequence
 In Unix, though rare, there is a possibility of clock drifting backwards (or
 forward) after a restart. In such rare scenarios, we might end up with a logical clock value less than the previously used value, this violates the fundamental requirement of monotonically increasing clock. To avoid such disasters, every logical clock tick is persisted using sequences (non-transactional). After a restart, the persisted sequence value is read and clock starts from that value, which will ensure that system starts the clock from where we left off.
 ### Psuedo code
    WC - Current Wall Clock in milliseconds
    HLC - Current Hybrid Logical Clock in shared
          memory
    MAX_COUNTER - Four million
    /* Tick the clock by 1 */
    IncrementClusterClock()
    {
       /* It's the counter that always ticks, once it reaches
          the maximum, reset the counter to 1 and increment
          the logical clock. */
        if (HLC.C == MAX_COUNTER)
        {
                HLC.LC++;
                HLC.C = 0;
                return;
        }
        HLC.C++;
    }
    /* Tick for each event, must increase monotonically */
    GetNextNodeClockValue()
    {
 	    IncrementClusterClock(HLC);
 	    /* From the incremented clock and current wall clock,
 	       pick which ever is highest */
 	    NextClock = MAX(HLC, WC);
 	    /* Save the NextClock value in both the shared memory
 	       and sequence */
 	    HLC = NextClock;
 	    SETVAL(pg_dist_clock_logical_seq, HLC);
 	}
 	/* Returns true if the clock1 is after clock2 */
    IsEventAfter(HLC1, HLC2)
    {
 	    IF (HLC1.LC != HLC2.LC)
 		    return (HLC1.LC > HLC2.LC);
 		ELSE
 			return (HLC1.C > HLC2.C);
    }
    /* Simply returns the highest node clock value among all
       nodes */
    GetHighestClockInTransaction()
    {
 	    For each node
 	    {
 		    NodeClock[N] = GetNextNodeClockValue();
 		}
 		/* Return the highest clock value of all the nodes */
 		return MAX(NodeClock[N]);
    }
    /* Adjust the local shared memory clock to the received
       value (RHLC) from the remote node */
 	AdjustClock(RHLC)
 	{
 		/* local clock is ahead or equal, do nothing */
        IF (HLC >= RHLC)
        {
          return;
        }
        /* Save the remote clockvalue in both the shared
           memory and sequence */
 	    HLC = RHLC;
 	    SETVAL(pg_dist_clock_logical_seq, HLC);
 	}
    /* All the nodes will adjust their clocks to the highest
       of the newly negotiated clock */
    AdjustClocksToTransactionHighest(HLC)
    {
 	    For each node
 	    {
 		    SendCommand ("AdjustClock(HLC)");
 		}
    }
    /* When citus_get_transaction_clock() UDF is invoked */
    PrepareAndSetTransactionClock()
    {
 	    /* Pick the highest logical clock value among all
 	       transaction-nodes */
        txnCLock = GetHighestClockInTransaction()
        /* Adjust all the nodes with the new clock value */
        AdjustClocksToTransactionHighest(txnCLock )
        return txnClock;
    }
    /* Initialize the clock value to the highest clock
       persisted in sequence */
    InitClockAtBoot()
    {
 	    /* Start with the current wall clock */
 	    HLC = WC;
 	    IF (SEQUENCE == 1)
 		/* clock never ticked on this node, start with the
 		   wall clock. */
 				return;
 		/* get the most recent clock ever used from disk */
 		persistedClock =
 		    NEXT_VAL(pg_dist_clock_logical_seq...)
 		    /* Start the clock with persisted value */
 		    AdjustLocalClock(persistedClock);
 	    }
    }
 #### Usage
 **Step 1**
 In the application, track every change of a transaction along with the unique transaction ID by calling UDF
 `get_current_transaction_id`()
    INSERT INTO track_table
    SET TransactionId =
    get_current_transaction_id(),
    operation = <insert/update/delete>,
    row_key = <>,
    ....;
 **Step 2**
 As the transaction is about to end, and before the COMMIT, capture the causal clock timestamp along with the transaction ID in a table
    INSERT INTO transaction_commit_clock
    (TransactionId, CommitClock, timestamp)
    SELECT
            citus_get_transaction_clock(),
            get_current_transaction_id(),
            now()
 **Step 3**
 How to get all the events in the causal order?
    SELECT tt.row_key, tt.operation
    FROM track_table tt,
         transaction_commit_clock cc
    WHERE tt.TransactionId = cc.TransactionId
    ORDER BY cc.CommitClock
 Events for an object
    SELECT tt.row_key, tt.operation
    FROM track_table tt,
         transaction_commit_clock cc
    WHERE tt.TransactionId = cc.TransactionId
    and row_key = $1 ORDER BY cc.CommitClock
 Events in the last one hour
    SELECT tt.row_key, tt.operation
    FROM track_table tt,
         transaction_commit_clockcc
    WHERE cc.timestamp >= now() - interval '1 hour'
    and tt.TransactionId = cc.TransactionId
 **Note**: In Citus we use 2PC, if any node goes down after the PREPARE and before the COMMIT, we might have changes partially committed. Citus tracks such transactions in **pg_dist_transaction** and eventually will be committed when the node becomes healthy, but when we track change-data from committed transactions of **transaction_commit_clock** we will miss the changes from a bad node.
 To address this issue, proposal is to have a new UDF #TBD, that freezes
 the clock and ensures that all the 2PCs are fully complete
 (i.e., **pg_dist_transaction** should be empty) and return the highest
 clock used. All transactions in `transaction_commit_clock` with
 timestamp below this returned clock are visible to the application. The
 exact nuances, such as frequency of calling such UDF, are still TBD.
 Caveat is, if the node and the 2PC takes long to fully recover, the
 visibility of the committed transactions might stall.
 ### Catalog pruning
 The data in **transaction_commit_clock** should be ephemeral data i.e., eventually rows have to automatically be deleted. Users can install a pg_cron job to prune the catalog regularly.
    delete from transaction_commit_clock
        where timestamp < now() - interval '7 days'
 ### Limitations of Citus
 Using this transaction commit clock ordering to build a secondary, that's a mirror copy of the original, may not be feasible at this time for the following reasons.
 Given that there is no well-defined order between concurrent distributed transactions in Citus, we cannot retroactively apply a transaction-order that leads to an exact replica of the primary unless we preserve the original object-level ordering as it happened on individual nodes.
 For instance, if a multi-shard insert (transaction A) happens concurrently with a multi-shard update (transaction B) and the WHERE clause of the update matches inserted rows in multiple shards, we could have a scenario in which only a subset of the inserted rows gets updated. Effectively, transaction A might happen before transaction B on node 1, while transaction B happens before transaction A on node 2. While unfortunate, we cannot simply claim changes made by transaction A happened first based on commit timestamps, because that would lead us reorder changes to the same object ID on node 2, which might lead to a different outcome when replayed.
 In such scenario, even if we use causal commit clock to order changes. It is essential that the order of modifications to an object matches the original order. Otherwise, you could have above scenarios where an insert happens before an update in the primary cluster, but the update happens before the insert. Replaying the changes would then lead to a different database.
 In absence of a coherent transaction-ordering semantics in distributed cluster, best we can do is ensure that changes to the same object are in the correct order and ensure exactly once delivery (correct pagination).
--- a/src/backend/distributed/clock/causal_clock.c
+++ b/src/backend/distributed/clock/causal_clock.c
@ -1,622 +0,0 @@
 /*-------------------------------------------------------------------------
 * causal_clock.c
 *
 * Core function defintions to implement hybrid logical clock.
 *
 * Copyright (c) Citus Data, Inc.
 *
 *-------------------------------------------------------------------------
 */
 #include <sys/time.h>
 #include "postgres.h"
 #include "fmgr.h"
 #include "funcapi.h"
 #include "libpq-fe.h"
 #include "miscadmin.h"
 #include "catalog/namespace.h"
 #include "commands/extension.h"
 #include "commands/sequence.h"
 #include "executor/spi.h"
 #include "nodes/pg_list.h"
 #include "postmaster/postmaster.h"
 #include "storage/ipc.h"
 #include "storage/lwlock.h"
 #include "storage/s_lock.h"
 #include "storage/shmem.h"
 #include "storage/spin.h"
 #include "utils/builtins.h"
 #include "utils/datum.h"
 #include "utils/numeric.h"
 #include "utils/typcache.h"
 #include "distributed/causal_clock.h"
 #include "distributed/citus_safe_lib.h"
 #include "distributed/coordinator_protocol.h"
 #include "distributed/listutils.h"
 #include "distributed/local_executor.h"
 #include "distributed/lock_graph.h"
 #include "distributed/metadata_cache.h"
 #include "distributed/placement_connection.h"
 #include "distributed/remote_commands.h"
 #define SAVE_AND_PERSIST(c) \
 	do { \
 		Oid savedUserId = InvalidOid; \
 		int savedSecurityContext = 0; \
 		LogicalClockShmem->clusterClockValue = *(c); \
 		GetUserIdAndSecContext(&savedUserId, &savedSecurityContext); \
 		SetUserIdAndSecContext(CitusExtensionOwner(), SECURITY_LOCAL_USERID_CHANGE); \
 		DirectFunctionCall2(setval_oid, \
 							ObjectIdGetDatum(DistClockLogicalSequenceId()), \
 							Int64GetDatum((c)->logical)); \
 		SetUserIdAndSecContext(savedUserId, savedSecurityContext); \
 	} while (0)
 PG_FUNCTION_INFO_V1(citus_get_node_clock);
 PG_FUNCTION_INFO_V1(citus_internal_adjust_local_clock_to_remote);
 PG_FUNCTION_INFO_V1(citus_is_clock_after);
 PG_FUNCTION_INFO_V1(citus_get_transaction_clock);
 /*
 * Current state of the logical clock
 */
 typedef enum ClockState
 {
 	CLOCKSTATE_INITIALIZED,
 	CLOCKSTATE_UNINITIALIZED
 } ClockState;
 /*
 * Holds the cluster clock variables in shared memory.
 */
 typedef struct LogicalClockShmemData
 {
 	NamedLWLockTranche namedLockTranche;
 	LWLock clockLock;
 	/* Current logical clock value of this node */
 	ClusterClock clusterClockValue;
 	/* Tracks initialization at boot */
 	ClockState clockInitialized;
 } LogicalClockShmemData;
 static shmem_startup_hook_type prev_shmem_startup_hook = NULL;
 static LogicalClockShmemData *LogicalClockShmem = NULL;
 static void AdjustLocalClock(ClusterClock *remoteClock);
 static void GetNextNodeClockValue(ClusterClock *nextClusterClockValue);
 static ClusterClock * GetHighestClockInTransaction(List *nodeConnectionList);
 static void AdjustClocksToTransactionHighest(List *nodeConnectionList,
 											 ClusterClock *transactionClockValue);
 static void InitClockAtFirstUse(void);
 static void IncrementClusterClock(ClusterClock *clusterClock);
 static ClusterClock * LargerClock(ClusterClock *clock1, ClusterClock *clock2);
 static ClusterClock * PrepareAndSetTransactionClock(void);
 bool EnableClusterClock = true;
 /*
 * GetEpochTimeAsClock returns the epoch value milliseconds used as logical
 * value in ClusterClock.
 */
 ClusterClock *
 GetEpochTimeAsClock(void)
 {
 	struct timeval tp = { 0 };
 	gettimeofday(&tp, NULL);
 	uint64 result = (uint64) (tp.tv_sec) * 1000;
 	result = result + (uint64) (tp.tv_usec) / 1000;
 	ClusterClock *epochClock = (ClusterClock *) palloc(sizeof(ClusterClock));
 	epochClock->logical = result;
 	epochClock->counter = 0;
 	return epochClock;
 }
 /*
 * LogicalClockShmemSize returns the size that should be allocated
 * in the shared memory for logical clock management.
 */
 size_t
 LogicalClockShmemSize(void)
 {
 	Size size = 0;
 	size = add_size(size, sizeof(LogicalClockShmemData));
 	return size;
 }
 /*
 * InitializeClusterClockMem reserves shared-memory space needed to
 * store LogicalClockShmemData, and sets the hook for initialization
 * of the same.
 */
 void
 InitializeClusterClockMem(void)
 {
 	prev_shmem_startup_hook = shmem_startup_hook;
 	shmem_startup_hook = LogicalClockShmemInit;
 }
 /*
 * LogicalClockShmemInit Allocates and initializes shared memory for
 * cluster clock related variables.
 */
 void
 LogicalClockShmemInit(void)
 {
 	bool alreadyInitialized = false;
 	LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
 	LogicalClockShmem = (LogicalClockShmemData *)
 						ShmemInitStruct("Logical Clock Shmem",
 										LogicalClockShmemSize(),
 										&alreadyInitialized);
 	if (!alreadyInitialized)
 	{
 		/* A zero value indicates that the clock is not adjusted yet */
 		memset(&LogicalClockShmem->clusterClockValue, 0, sizeof(ClusterClock));
 		LogicalClockShmem->namedLockTranche.trancheName = "Cluster Clock Setup Tranche";
 		LogicalClockShmem->namedLockTranche.trancheId = LWLockNewTrancheId();
 		LWLockRegisterTranche(LogicalClockShmem->namedLockTranche.trancheId,
 							  LogicalClockShmem->namedLockTranche.trancheName);
 		LWLockInitialize(&LogicalClockShmem->clockLock,
 						 LogicalClockShmem->namedLockTranche.trancheId);
 		LogicalClockShmem->clockInitialized = CLOCKSTATE_UNINITIALIZED;
 	}
 	LWLockRelease(AddinShmemInitLock);
 	if (prev_shmem_startup_hook != NULL)
 	{
 		prev_shmem_startup_hook();
 	}
 }
 /*
 * IncrementClusterClock increments the ClusterClock by 1.
 */
 static void
 IncrementClusterClock(ClusterClock *clusterClock)
 {
 	/*
 	 * It's the counter that always ticks, once it reaches the maximum, reset
 	 * the counter to 1 and increment the logical clock.
 	 */
 	if (clusterClock->counter == MAX_COUNTER)
 	{
 		clusterClock->logical++;
 		clusterClock->counter = 0;
 		return;
 	}
 	clusterClock->counter++;
 }
 /*
 * LargerClock compares two ClusterClock(s) and returns pointer to the larger one.
 * Note: If equal or one of the clock is NULL, non NULL clock is copied.
 */
 static ClusterClock *
 LargerClock(ClusterClock *clock1, ClusterClock *clock2)
 {
 	/* Check if one of the paramater is NULL */
 	if (!clock1 || !clock2)
 	{
 		Assert(clock1 || clock2);
 		return (!clock1 ? clock2 : clock1);
 	}
 	if (cluster_clock_cmp_internal(clock1, clock2) > 0)
 	{
 		return clock1;
 	}
 	else
 	{
 		return clock2;
 	}
 }
 /*
 * GetNextNodeClock implements the internal guts of the UDF citus_get_node_clock()
 */
 static void
 GetNextNodeClockValue(ClusterClock *nextClusterClockValue)
 {
 	static bool isClockInitChecked = false; /* serves as a local cache */
 	ClusterClock *epochValue = GetEpochTimeAsClock();
 	/* If this backend already checked for initialization, skip it */
 	if (!isClockInitChecked)
 	{
 		InitClockAtFirstUse();
 		/* We reach here only if CLOCKSTATE_INITIALIZED, all other cases error out. */
 		isClockInitChecked = true;
 	}
 	LWLockAcquire(&LogicalClockShmem->clockLock, LW_EXCLUSIVE);
 	Assert(LogicalClockShmem->clockInitialized == CLOCKSTATE_INITIALIZED);
 	/* Tick the clock */
 	IncrementClusterClock(&LogicalClockShmem->clusterClockValue);
 	/* Pick the larger of the two, wallclock and logical clock. */
 	ClusterClock *clockValue = LargerClock(&LogicalClockShmem->clusterClockValue,
 										   epochValue);
 	/*
 	 * Save the returned value in both the shared memory and sequences.
 	 */
 	SAVE_AND_PERSIST(clockValue);
 	LWLockRelease(&LogicalClockShmem->clockLock);
 	/* Return the clock */
 	*nextClusterClockValue = *clockValue;
 }
 /*
 * AdjustLocalClock Adjusts the local shared memory clock to the
 * received value from the remote node.
 */
 void
 AdjustLocalClock(ClusterClock *remoteClock)
 {
 	LWLockAcquire(&LogicalClockShmem->clockLock, LW_EXCLUSIVE);
 	ClusterClock *localClock = &LogicalClockShmem->clusterClockValue;
 	/* local clock is ahead or equal, do nothing */
 	if (cluster_clock_cmp_internal(localClock, remoteClock) >= 0)
 	{
 		LWLockRelease(&LogicalClockShmem->clockLock);
 		return;
 	}
 	SAVE_AND_PERSIST(remoteClock);
 	LWLockRelease(&LogicalClockShmem->clockLock);
 	ereport(DEBUG1, (errmsg("adjusted to remote clock: "
 							"<logical(%lu) counter(%u)>",
 							remoteClock->logical,
 							remoteClock->counter)));
 }
 /*
 * GetHighestClockInTransaction takes the connection list of participating nodes in the
 * current transaction and polls the logical clock value of all the nodes. Returns the
 * highest logical clock value of all the nodes in the current distributed transaction,
 * which may be used as commit order for individual objects in the transaction.
 */
 static ClusterClock *
 GetHighestClockInTransaction(List *nodeConnectionList)
 {
 	MultiConnection *connection = NULL;
 	foreach_declared_ptr(connection, nodeConnectionList)
 	{
 		int querySent =
 			SendRemoteCommand(connection, "SELECT citus_get_node_clock();");
 		if (querySent == 0)
 		{
 			ReportConnectionError(connection, ERROR);
 		}
 	}
 	/* Check for the current node */
 	ClusterClock *globalClockValue = (ClusterClock *) palloc(sizeof(ClusterClock));
 	GetNextNodeClockValue(globalClockValue);
 	ereport(DEBUG1, (errmsg("node(%u) transaction clock %lu:%u",
 							PostPortNumber, globalClockValue->logical,
 							globalClockValue->counter)));
 	/* fetch the results and pick the highest clock value of all the nodes */
 	foreach_declared_ptr(connection, nodeConnectionList)
 	{
 		bool raiseInterrupts = true;
 		if (PQstatus(connection->pgConn) != CONNECTION_OK)
 		{
 			ereport(ERROR, (errmsg("connection to %s:%d failed when "
 								   "fetching logical clock value",
 								   connection->hostname, connection->port)));
 		}
 		PGresult *result = GetRemoteCommandResult(connection, raiseInterrupts);
 		if (!IsResponseOK(result))
 		{
 			ReportResultError(connection, result, ERROR);
 		}
 		ClusterClock *nodeClockValue = ParseClusterClockPGresult(result, 0, 0);
 		ereport(DEBUG1, (errmsg("node(%u) transaction clock %lu:%u",
 								connection->port, nodeClockValue->logical,
 								nodeClockValue->counter)));
 		globalClockValue = LargerClock(globalClockValue, nodeClockValue);
 		PQclear(result);
 		ForgetResults(connection);
 	}
 	ereport(DEBUG1, (errmsg("final global transaction clock %lu:%u",
 							globalClockValue->logical,
 							globalClockValue->counter)));
 	return globalClockValue;
 }
 /*
 * AdjustClocksToTransactionHighest Sets the clock value of all the nodes, participated
 * in the PREPARE of the transaction, to the highest clock value of all the nodes.
 */
 static void
 AdjustClocksToTransactionHighest(List *nodeConnectionList,
 								 ClusterClock *transactionClockValue)
 {
 	StringInfo queryToSend = makeStringInfo();
 	/* Set the clock value on participating worker nodes */
 	appendStringInfo(queryToSend,
 					 "SELECT citus_internal.adjust_local_clock_to_remote"
 					 "('(%lu, %u)'::pg_catalog.cluster_clock);",
 					 transactionClockValue->logical, transactionClockValue->counter);
 	ExecuteRemoteCommandInConnectionList(nodeConnectionList, queryToSend->data);
 	AdjustLocalClock(transactionClockValue);
 }
 /*
 * PrepareAndSetTransactionClock polls all the transaction-nodes for their respective clocks,
 * picks the highest clock and returns it via UDF citus_get_transaction_clock. All the nodes
 * will now move to this newly negotiated clock.
 */
 static ClusterClock *
 PrepareAndSetTransactionClock(void)
 {
 	if (!EnableClusterClock)
 	{
 		/* citus.enable_cluster_clock is false */
 		ereport(WARNING, (errmsg("GUC enable_cluster_clock is off")));
 		return NULL;
 	}
 	dlist_iter iter;
 	List *transactionNodeList = NIL;
 	List *nodeList = NIL;
 	/* Prepare the connection list */
 	dlist_foreach(iter, &InProgressTransactions)
 	{
 		MultiConnection *connection = dlist_container(MultiConnection, transactionNode,
 													  iter.cur);
 		WorkerNode *workerNode = FindWorkerNode(connection->hostname, connection->port);
 		if (!workerNode)
 		{
 			ereport(WARNING, errmsg("Worker node is missing"));
 			continue;
 		}
 		/* Skip the node if we already in the list */
 		if (list_member_int(nodeList, workerNode->groupId))
 		{
 			continue;
 		}
 		RemoteTransaction *transaction = &connection->remoteTransaction;
 		Assert(transaction->transactionState != REMOTE_TRANS_NOT_STARTED);
 		/* Skip a transaction that failed */
 		if (transaction->transactionFailed)
 		{
 			continue;
 		}
 		nodeList = lappend_int(nodeList, workerNode->groupId);
 		transactionNodeList = lappend(transactionNodeList, connection);
 	}
 	/* Pick the highest logical clock value among all transaction-nodes */
 	ClusterClock *transactionClockValue =
 		GetHighestClockInTransaction(transactionNodeList);
 	/* Adjust all the nodes with the new clock value */
 	AdjustClocksToTransactionHighest(transactionNodeList, transactionClockValue);
 	return transactionClockValue;
 }
 /*
 * InitClockAtFirstUse Initializes the shared memory clock value to the highest clock
 * persisted. This will protect from any clock drifts.
 */
 static void
 InitClockAtFirstUse(void)
 {
 	LWLockAcquire(&LogicalClockShmem->clockLock, LW_EXCLUSIVE);
 	/* Avoid repeated and parallel initialization */
 	if (LogicalClockShmem->clockInitialized == CLOCKSTATE_INITIALIZED)
 	{
 		LWLockRelease(&LogicalClockShmem->clockLock);
 		return;
 	}
 	if (DistClockLogicalSequenceId() == InvalidOid)
 	{
 		ereport(ERROR, (errmsg("Clock related sequence is missing")));
 	}
 	/* Start with the wall clock value */
 	ClusterClock *epochValue = GetEpochTimeAsClock();
 	LogicalClockShmem->clusterClockValue = *epochValue;
 	/*  Retrieve the highest clock value persisted in the sequence */
 	ClusterClock persistedMaxClock = { 0 };
 	/*
 	 * We will get one more than the persisted value, but that's harmless and
 	 * also very _crucial_ in below scenarios
 	 *
 	 * 1) As sequences are not transactional, this will protect us from crashes
 	 *    after the logical increment and before the counter increment.
 	 *
 	 * 2) If a clock drifts backwards, we should always start one clock above
 	 *    the previous value, though we are not persisting the counter as the
 	 *    logical value supersedes the counter, a simple increment of it will
 	 *    protect us.
 	 *
 	 * Note: The first (and every 32nd) call to nextval() consumes 32 values in the
 	 * WAL. This is an optimization that postgres does to only have to write a WAL
 	 * entry every 32 invocations. Normally this is harmless, however, if the database
 	 * gets in a crashloop it could outrun the wall clock, if the database crashes at
 	 * a higher rate than once every 32 seconds.
 	 *
 	 */
 	Oid saveUserId = InvalidOid;
 	int savedSecurityCtx = 0;
 	GetUserIdAndSecContext(&saveUserId, &savedSecurityCtx);
 	SetUserIdAndSecContext(CitusExtensionOwner(), SECURITY_LOCAL_USERID_CHANGE);
 	persistedMaxClock.logical =
 		DirectFunctionCall1(nextval_oid, ObjectIdGetDatum(DistClockLogicalSequenceId()));
 	SetUserIdAndSecContext(saveUserId, savedSecurityCtx);
 	/*
 	 * Sequence 1 indicates no prior clock timestamps on this server, retain
 	 * the wall clock i.e. no adjustment needed.
 	 */
 	if (persistedMaxClock.logical != 1)
 	{
 		ereport(DEBUG1, (errmsg("adjusting the clock with persisted value: "
 								"<logical(%lu) and counter(%u)>",
 								persistedMaxClock.logical,
 								persistedMaxClock.counter)));
 		/*
 		 * Adjust the local clock according to the most recent
 		 * clock stamp value persisted in the catalog.
 		 */
 		if (cluster_clock_cmp_internal(&persistedMaxClock, epochValue) > 0)
 		{
 			SAVE_AND_PERSIST(&persistedMaxClock);
 			ereport(NOTICE, (errmsg("clock drifted backwards, adjusted to: "
 									"<logical(%lu) counter(%u)>",
 									persistedMaxClock.logical,
 									persistedMaxClock.counter)));
 		}
 	}
 	LogicalClockShmem->clockInitialized = CLOCKSTATE_INITIALIZED;
 	LWLockRelease(&LogicalClockShmem->clockLock);
 }
 /*
 * citus_get_node_clock() is an UDF that returns a monotonically increasing
 * logical clock. Clock guarantees to never go back in value after restarts, and
 * makes best attempt to keep the value close to unix epoch time in milliseconds.
 */
 Datum
 citus_get_node_clock(PG_FUNCTION_ARGS)
 {
 	CheckCitusVersion(ERROR);
 	ClusterClock *nodeClockValue = (ClusterClock *) palloc(sizeof(ClusterClock));
 	GetNextNodeClockValue(nodeClockValue);
 	PG_RETURN_POINTER(nodeClockValue);
 }
 /*
 * citus_internal_adjust_local_clock_to_remote is an internal UDF to adjust
 * the local clock to the highest in the cluster.
 */
 Datum
 citus_internal_adjust_local_clock_to_remote(PG_FUNCTION_ARGS)
 {
 	CheckCitusVersion(ERROR);
 	ClusterClock *remoteClock = (ClusterClock *) PG_GETARG_POINTER(0);
 	AdjustLocalClock(remoteClock);
 	PG_RETURN_VOID();
 }
 /*
 * citus_is_clock_after is an UDF that accepts logical clock timestamps of
 * two causally related events and returns true if the argument1 happened
 * before argument2.
 */
 Datum
 citus_is_clock_after(PG_FUNCTION_ARGS)
 {
 	CheckCitusVersion(ERROR);
 	/* Fetch both the arguments */
 	ClusterClock *clock1 = (ClusterClock *) PG_GETARG_POINTER(0);
 	ClusterClock *clock2 = (ClusterClock *) PG_GETARG_POINTER(1);
 	ereport(DEBUG1, (errmsg(
 						 "clock1 @ LC:%lu, C:%u, "
 						 "clock2 @ LC:%lu, C:%u",
 						 clock1->logical, clock1->counter,
 						 clock2->logical, clock2->counter)));
 	bool result = (cluster_clock_cmp_internal(clock1, clock2) > 0);
 	PG_RETURN_BOOL(result);
 }
 /*
 * citus_get_transaction_clock() is an UDF that returns a transaction timestamp
 * logical clock. Clock returned is the maximum of all transaction-nodes and the
 * all the nodes adjust to the this new clock value.
 */
 Datum
 citus_get_transaction_clock(PG_FUNCTION_ARGS)
 {
 	CheckCitusVersion(ERROR);
 	ClusterClock *clusterClockValue = PrepareAndSetTransactionClock();
 	PG_RETURN_POINTER(clusterClockValue);
 }
--- a/src/backend/distributed/commands/alter_table.c
+++ b/src/backend/distributed/commands/alter_table.c
@ -34,16 +34,9 @@
 #include "catalog/pg_am.h"
 #include "catalog/pg_depend.h"
 #include "catalog/pg_rewrite_d.h"
 #include "commands/defrem.h"
 #include "executor/spi.h"
 #include "nodes/pg_list.h"
 #include "utils/builtins.h"
 #include "utils/lsyscache.h"
 #include "utils/syscache.h"
 #include "columnar/columnar.h"
 #include "columnar/columnar_tableam.h"
-
+#include "commands/defrem.h"
 #include "distributed/colocation_utils.h"
 #include "distributed/commands.h"
 #include "distributed/commands/utility_hook.h"
@ -60,15 +53,17 @@
 #include "distributed/multi_executor.h"
 #include "distributed/multi_logical_planner.h"
 #include "distributed/multi_partitioning_utils.h"
 #include "distributed/namespace_utils.h"
 #include "distributed/reference_table_utils.h"
 #include "distributed/relation_access_tracking.h"
 #include "distributed/replication_origin_session_utils.h"
 #include "distributed/shard_utils.h"
 #include "distributed/shared_library_init.h"
-#include "distributed/tenant_schema_metadata.h"
+#include "distributed/shard_utils.h"
 #include "distributed/worker_protocol.h"
 #include "distributed/worker_transaction.h"
 #include "executor/spi.h"
 #include "nodes/pg_list.h"
 #include "utils/builtins.h"
 #include "utils/lsyscache.h"
 #include "utils/syscache.h"
 /* Table Conversion Types */
@ -188,7 +183,6 @@ static TableConversionReturn * AlterDistributedTable(TableConversionParameters *
 static TableConversionReturn * AlterTableSetAccessMethod(
 	TableConversionParameters *params);
 static TableConversionReturn * ConvertTable(TableConversionState *con);
 static TableConversionReturn * ConvertTableInternal(TableConversionState *con);
 static bool SwitchToSequentialAndLocalExecutionIfShardNameTooLong(char *relationName,
 																  char *longestShardName);
 static void DropIndexesNotSupportedByColumnar(Oid relationId,
@ -200,7 +194,6 @@ static void EnsureTableNotReferencing(Oid relationId, char conversionType);
 static void EnsureTableNotReferenced(Oid relationId, char conversionType);
 static void EnsureTableNotForeign(Oid relationId);
 static void EnsureTableNotPartition(Oid relationId);
 static void ErrorIfColocateWithTenantTable(char *colocateWith);
 static TableConversionState * CreateTableConversion(TableConversionParameters *params);
 static void CreateDistributedTableLike(TableConversionState *con);
 static void CreateCitusTableLike(TableConversionState *con);
@ -209,9 +202,12 @@ static void ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommand
 static bool HasAnyGeneratedStoredColumns(Oid relationId);
 static List * GetNonGeneratedStoredColumnNameList(Oid relationId);
 static void CheckAlterDistributedTableConversionParameters(TableConversionState *con);
-static char * CreateWorkerChangeSequenceDependencyCommand(char *qualifiedSequeceName,
+static char * CreateWorkerChangeSequenceDependencyCommand(char *sequenceSchemaName,
-														  char *qualifiedSourceName,
+														  char *sequenceName,
-														  char *qualifiedTargetName);
+														  char *sourceSchemaName,
 														  char *sourceName,
 														  char *targetSchemaName,
 														  char *targetName);
 static void ErrorIfMatViewSizeExceedsTheLimit(Oid matViewOid);
 static char * CreateMaterializedViewDDLCommand(Oid matViewOid);
 static char * GetAccessMethodForMatViewIfExists(Oid viewOid);
@ -219,10 +215,7 @@ static bool WillRecreateForeignKeyToReferenceTable(Oid relationId,
 												   CascadeToColocatedOption cascadeOption);
 static void WarningsForDroppingForeignKeysWithDistributedTables(Oid relationId);
 static void ErrorIfUnsupportedCascadeObjects(Oid relationId);
 static List * WrapTableDDLCommands(List *commandStrings);
 static bool DoesCascadeDropUnsupportedObject(Oid classId, Oid id, HTAB *nodeMap);
 static TableConversionReturn * CopyTableConversionReturnIntoCurrentContext(
 	TableConversionReturn *tableConversionReturn);
 PG_FUNCTION_INFO_V1(undistribute_table);
 PG_FUNCTION_INFO_V1(alter_distributed_table);
@ -249,8 +242,7 @@ undistribute_table(PG_FUNCTION_ARGS)
 	TableConversionParameters params = {
 		.relationId = relationId,
-		.cascadeViaForeignKeys = cascadeViaForeignKeys,
+		.cascadeViaForeignKeys = cascadeViaForeignKeys
 		.bypassTenantCheck = false
 	};
 	UndistributeTable(&params);
@ -363,124 +355,6 @@ worker_change_sequence_dependency(PG_FUNCTION_ARGS)
 }
 /*
 * DropFKeysAndUndistributeTable drops all foreign keys that relation with
 * relationId is involved then undistributes it.
 * Note that as UndistributeTable changes relationId of relation, this
 * function also returns new relationId of relation.
 * Also note that callers are responsible for storing & recreating foreign
 * keys to be dropped if needed.
 */
 Oid
 DropFKeysAndUndistributeTable(Oid relationId)
 {
 	DropFKeysRelationInvolvedWithTableType(relationId, INCLUDE_ALL_TABLE_TYPES);
 	/* store them before calling UndistributeTable as it changes relationId */
 	char *relationName = get_rel_name(relationId);
 	Oid schemaId = get_rel_namespace(relationId);
 	/* suppress notices messages not to be too verbose */
 	TableConversionParameters params = {
 		.relationId = relationId,
 		.cascadeViaForeignKeys = false,
 		.suppressNoticeMessages = true
 	};
 	UndistributeTable(&params);
 	Oid newRelationId = get_relname_relid(relationName, schemaId);
 	/*
 	 * We don't expect this to happen but to be on the safe side let's error
 	 * out here.
 	 */
 	EnsureRelationExists(newRelationId);
 	return newRelationId;
 }
 /*
 * UndistributeTables undistributes given relations. It first collects all foreign keys
 * to recreate them after the undistribution. Then, drops the foreign keys and
 * undistributes the relations. Finally, it recreates foreign keys.
 */
 void
 UndistributeTables(List *relationIdList)
 {
 	/*
 	 * Collect foreign keys for recreation and then drop fkeys and undistribute
 	 * tables.
 	 */
 	List *originalForeignKeyRecreationCommands = NIL;
 	Oid relationId = InvalidOid;
 	foreach_declared_oid(relationId, relationIdList)
 	{
 		List *fkeyCommandsForRelation =
 			GetFKeyCreationCommandsRelationInvolvedWithTableType(relationId,
 																 INCLUDE_ALL_TABLE_TYPES);
 		originalForeignKeyRecreationCommands = list_concat(
 			originalForeignKeyRecreationCommands, fkeyCommandsForRelation);
 		DropFKeysAndUndistributeTable(relationId);
 	}
 	/* We can skip foreign key validations as we are sure about them at start */
 	bool skip_validation = true;
 	ExecuteForeignKeyCreateCommandList(originalForeignKeyRecreationCommands,
 									   skip_validation);
 }
 /*
 * EnsureUndistributeTenantTableSafe ensures that it is safe to undistribute a tenant table.
 */
 void
 EnsureUndistributeTenantTableSafe(Oid relationId, const char *operationName)
 {
 	Oid schemaId = get_rel_namespace(relationId);
 	Assert(IsTenantSchema(schemaId));
 	/* We only allow undistribute while altering schema */
 	if (strcmp(operationName, TenantOperationNames[TENANT_SET_SCHEMA]) != 0)
 	{
 		ErrorIfTenantTable(relationId, operationName);
 	}
 	char *tableName = get_rel_name(relationId);
 	char *schemaName = get_namespace_name(schemaId);
 	/*
 	 * Partition table cannot be undistributed. Otherwise, its parent table would still
 	 * be a tenant table whereas partition table would be a local table.
 	 */
 	if (PartitionTable(relationId))
 	{
 		ereport(ERROR, (errmsg("%s is not allowed for partition table %s in distributed "
 							   "schema %s", operationName, tableName, schemaName),
 						errdetail("partition table should be under the same distributed "
 								  "schema as its parent and be a "
 								  "distributed schema table.")));
 	}
 	/*
 	 * When table is referenced by or referencing to a table in the same tenant
 	 * schema, we should disallow undistributing the table since we do not allow
 	 * foreign keys from/to Citus local or Postgres local table to/from distributed
 	 * schema.
 	 */
 	List *fkeyCommandsWithSingleShardTables =
 		GetFKeyCreationCommandsRelationInvolvedWithTableType(
 			relationId, INCLUDE_SINGLE_SHARD_TABLES);
 	if (fkeyCommandsWithSingleShardTables != NIL)
 	{
 		ereport(ERROR, (errmsg("%s is not allowed for table %s in distributed schema %s",
 							   operationName, tableName, schemaName),
 						errdetail("distributed schemas cannot have foreign keys from/to "
 								  "local tables or different schema")));
 	}
 }
 /*
 * UndistributeTable undistributes the given table. It uses ConvertTable function to
 * create a new local table and move everything to that table.
@ -501,14 +375,6 @@ UndistributeTable(TableConversionParameters *params)
 							   "because the table is not distributed")));
 	}
 	Oid schemaId = get_rel_namespace(params->relationId);
 	if (!params->bypassTenantCheck && IsTenantSchema(schemaId) &&
 		IsCitusTableType(params->relationId, SINGLE_SHARD_DISTRIBUTED))
 	{
 		EnsureUndistributeTenantTableSafe(params->relationId,
 										  TenantOperationNames[TENANT_UNDISTRIBUTE_TABLE]);
 	}
 	if (!params->cascadeViaForeignKeys)
 	{
 		EnsureTableNotReferencing(params->relationId, UNDISTRIBUTE_TABLE);
@ -536,11 +402,7 @@ UndistributeTable(TableConversionParameters *params)
 	params->conversionType = UNDISTRIBUTE_TABLE;
 	params->shardCountIsNull = true;
 	TableConversionState *con = CreateTableConversion(params);
-
+	return ConvertTable(con);
 	SetupReplicationOriginLocalSession();
 	TableConversionReturn *conv = ConvertTable(con);
 	ResetReplicationOriginLocalSession();
 	return conv;
 }
@ -564,9 +426,6 @@ AlterDistributedTable(TableConversionParameters *params)
 							   "is not distributed")));
 	}
 	ErrorIfTenantTable(params->relationId, TenantOperationNames[TENANT_ALTER_TABLE]);
 	ErrorIfColocateWithTenantTable(params->colocateWith);
 	EnsureTableNotForeign(params->relationId);
 	EnsureTableNotPartition(params->relationId);
 	EnsureHashDistributedTable(params->relationId);
@ -582,7 +441,6 @@ AlterDistributedTable(TableConversionParameters *params)
 		ereport(DEBUG1, (errmsg("setting multi shard modify mode to sequential")));
 		SetLocalMultiShardModifyModeToSequential();
 	}
 	return ConvertTable(con);
 }
@ -609,11 +467,8 @@ AlterTableSetAccessMethod(TableConversionParameters *params)
 	EnsureTableNotReferencing(params->relationId, ALTER_TABLE_SET_ACCESS_METHOD);
 	EnsureTableNotReferenced(params->relationId, ALTER_TABLE_SET_ACCESS_METHOD);
 	EnsureTableNotForeign(params->relationId);
-
+	if (IsCitusTableType(params->relationId, DISTRIBUTED_TABLE))
 	if (!IsCitusTableType(params->relationId, SINGLE_SHARD_DISTRIBUTED) &&
 		IsCitusTableType(params->relationId, DISTRIBUTED_TABLE))
 	{
 		/* we do not support non-hash distributed tables, except single shard tables */
 		EnsureHashDistributedTable(params->relationId);
 	}
@ -622,11 +477,6 @@ AlterTableSetAccessMethod(TableConversionParameters *params)
 		ereport(ERROR, (errmsg("you cannot alter access method of a partitioned table")));
 	}
 	if (get_rel_relkind(params->relationId) == RELKIND_VIEW)
 	{
 		ereport(ERROR, (errmsg("you cannot alter access method of a view")));
 	}
 	if (PartitionTable(params->relationId) &&
 		IsCitusTableType(params->relationId, DISTRIBUTED_TABLE))
 	{
@ -656,9 +506,9 @@ AlterTableSetAccessMethod(TableConversionParameters *params)
 /*
- * ConvertTableInternal is used for converting a table into a new table with different
+ * ConvertTable is used for converting a table into a new table with different properties.
- * properties. The conversion is done by creating a new table, moving everything to the
+ * The conversion is done by creating a new table, moving everything to the new table and
- * new table and dropping the old one. So the oid of the table is not preserved.
+ * dropping the old one. So the oid of the table is not preserved.
 *
 * The new table will have the same name, columns and rows. It will also have partitions,
 * views, sequences of the old table. Finally it will have everything created by
@ -677,7 +527,7 @@ AlterTableSetAccessMethod(TableConversionParameters *params)
 * in case you add a new way to return from this function.
 */
 TableConversionReturn *
-ConvertTableInternal(TableConversionState *con)
+ConvertTable(TableConversionState *con)
 {
 	InTableTypeConversionFunctionCall = true;
@ -715,11 +565,8 @@ ConvertTableInternal(TableConversionState *con)
 	char *newAccessMethod = con->accessMethod ? con->accessMethod :
 							con->originalAccessMethod;
 	IncludeSequenceDefaults includeSequenceDefaults = NEXTVAL_SEQUENCE_DEFAULTS;
 	IncludeIdentities includeIdentity = INCLUDE_IDENTITY;
 	List *preLoadCommands = GetPreLoadTableCreationCommands(con->relationId,
 															includeSequenceDefaults,
 															includeIdentity,
 															newAccessMethod);
 	if (con->accessMethod && strcmp(con->accessMethod, "columnar") == 0)
@ -740,18 +587,9 @@ ConvertTableInternal(TableConversionState *con)
 	List *justBeforeDropCommands = NIL;
 	List *attachPartitionCommands = NIL;
-	List *createViewCommands = GetViewCreationCommandsOfTable(con->relationId);
+	postLoadCommands =
-
+		list_concat(postLoadCommands,
-	postLoadCommands = list_concat(postLoadCommands,
+					GetViewCreationTableDDLCommandsOfTable(con->relationId));
 								   WrapTableDDLCommands(createViewCommands));
 	/* need to add back to publications after dropping the original table */
 	bool isAdd = true;
 	List *alterPublicationCommands =
 		GetAlterPublicationDDLCommandsForTable(con->relationId, isAdd);
 	postLoadCommands = list_concat(postLoadCommands,
 								   WrapTableDDLCommands(alterPublicationCommands));
 	List *foreignKeyCommands = NIL;
 	if (con->conversionType == ALTER_DISTRIBUTED_TABLE)
@ -788,21 +626,19 @@ ConvertTableInternal(TableConversionState *con)
 		justBeforeDropCommands = lappend(justBeforeDropCommands, detachFromParentCommand);
 	}
 	char *qualifiedRelationName = quote_qualified_identifier(con->schemaName,
 															 con->relationName);
 	if (PartitionedTable(con->relationId))
 	{
 		if (!con->suppressNoticeMessages)
 		{
 			ereport(NOTICE, (errmsg("converting the partitions of %s",
-									qualifiedRelationName)));
+									quote_qualified_identifier(con->schemaName,
 															   con->relationName))));
 		}
 		List *partitionList = PartitionList(con->relationId);
 		Oid partitionRelationId = InvalidOid;
-		foreach_declared_oid(partitionRelationId, partitionList)
+		foreach_oid(partitionRelationId, partitionList)
 		{
 			char *tableQualifiedName = generate_qualified_relation_name(
 				partitionRelationId);
@ -869,11 +705,13 @@ ConvertTableInternal(TableConversionState *con)
 	if (!con->suppressNoticeMessages)
 	{
-		ereport(NOTICE, (errmsg("creating a new table for %s", qualifiedRelationName)));
+		ereport(NOTICE, (errmsg("creating a new table for %s",
 								quote_qualified_identifier(con->schemaName,
 														   con->relationName))));
 	}
 	TableDDLCommand *tableCreationCommand = NULL;
-	foreach_declared_ptr(tableCreationCommand, preLoadCommands)
+	foreach_ptr(tableCreationCommand, preLoadCommands)
 	{
 		Assert(CitusIsA(tableCreationCommand, TableDDLCommand));
@ -947,28 +785,16 @@ ConvertTableInternal(TableConversionState *con)
 				 con->suppressNoticeMessages);
 	TableDDLCommand *tableConstructionCommand = NULL;
-	foreach_declared_ptr(tableConstructionCommand, postLoadCommands)
+	foreach_ptr(tableConstructionCommand, postLoadCommands)
 	{
 		Assert(CitusIsA(tableConstructionCommand, TableDDLCommand));
 		char *tableConstructionSQL = GetTableDDLCommand(tableConstructionCommand);
 		ExecuteQueryViaSPI(tableConstructionSQL, SPI_OK_UTILITY);
 	}
 	/*
 	 * when there are many partitions, each call to ProcessUtilityParseTree
 	 * accumulates used memory. Free context after each call.
 	 */
 	MemoryContext citusPerPartitionContext =
 		AllocSetContextCreate(CurrentMemoryContext,
 							  "citus_per_partition_context",
 							  ALLOCSET_DEFAULT_SIZES);
 	MemoryContext oldContext = MemoryContextSwitchTo(citusPerPartitionContext);
 	char *attachPartitionCommand = NULL;
-	foreach_declared_ptr(attachPartitionCommand, attachPartitionCommands)
+	foreach_ptr(attachPartitionCommand, attachPartitionCommands)
 	{
 		MemoryContextReset(citusPerPartitionContext);
 		Node *parseTree = ParseTreeNode(attachPartitionCommand);
 		ProcessUtilityParseTree(parseTree, attachPartitionCommand,
@ -976,9 +802,6 @@ ConvertTableInternal(TableConversionState *con)
 								NULL, None_Receiver, NULL);
 	}
 	MemoryContextSwitchTo(oldContext);
 	MemoryContextDelete(citusPerPartitionContext);
 	if (isPartitionTable)
 	{
 		ExecuteQueryViaSPI(attachToParentCommand, SPI_OK_UTILITY);
@ -990,12 +813,14 @@ ConvertTableInternal(TableConversionState *con)
 		/* For now we only support cascade to colocation for alter_distributed_table UDF */
 		Assert(con->conversionType == ALTER_DISTRIBUTED_TABLE);
-		foreach_declared_oid(colocatedTableId, con->colocatedTableList)
+		foreach_oid(colocatedTableId, con->colocatedTableList)
 		{
 			if (colocatedTableId == con->relationId)
 			{
 				continue;
 			}
 			char *qualifiedRelationName = quote_qualified_identifier(con->schemaName,
 																	 con->relationName);
 			TableConversionParameters cascadeParam = {
 				.relationId = colocatedTableId,
@ -1018,7 +843,7 @@ ConvertTableInternal(TableConversionState *con)
 		if (con->cascadeToColocated != CASCADE_TO_COLOCATED_NO_ALREADY_CASCADED)
 		{
 			char *foreignKeyCommand = NULL;
-			foreach_declared_ptr(foreignKeyCommand, foreignKeyCommands)
+			foreach_ptr(foreignKeyCommand, foreignKeyCommands)
 			{
 				ExecuteQueryViaSPI(foreignKeyCommand, SPI_OK_UTILITY);
 			}
@ -1036,77 +861,10 @@ ConvertTableInternal(TableConversionState *con)
 	SetLocalEnableLocalReferenceForeignKeys(oldEnableLocalReferenceForeignKeys);
 	InTableTypeConversionFunctionCall = false;
 	return ret;
 }
 /*
 * CopyTableConversionReturnIntoCurrentContext copies given tableConversionReturn
 * into CurrentMemoryContext.
 */
 static TableConversionReturn *
 CopyTableConversionReturnIntoCurrentContext(TableConversionReturn *tableConversionReturn)
 {
 	TableConversionReturn *tableConversionReturnCopy = NULL;
 	if (tableConversionReturn)
 	{
 		tableConversionReturnCopy = palloc0(sizeof(TableConversionReturn));
 		List *copyForeignKeyCommands = NIL;
 		char *foreignKeyCommand = NULL;
 		foreach_declared_ptr(foreignKeyCommand, tableConversionReturn->foreignKeyCommands)
 		{
 			char *copyForeignKeyCommand = MemoryContextStrdup(CurrentMemoryContext,
 															  foreignKeyCommand);
 			copyForeignKeyCommands = lappend(copyForeignKeyCommands,
 											 copyForeignKeyCommand);
 		}
 		tableConversionReturnCopy->foreignKeyCommands = copyForeignKeyCommands;
 	}
 	return tableConversionReturnCopy;
 }
 /*
 * ConvertTable is a wrapper for ConvertTableInternal to persist only
 * TableConversionReturn and delete all other allocations.
 */
 static TableConversionReturn *
 ConvertTable(TableConversionState *con)
 {
 	/*
 	 * We do not allow alter_distributed_table and undistribute_table operations
 	 * for tables with identity columns. This is because we do not have a proper way
 	 * of keeping sequence states consistent across the cluster.
 	 */
 	ErrorIfTableHasIdentityColumn(con->relationId);
 	/*
 	 * when there are many partitions or colocated tables, memory usage is
 	 * accumulated. Free context for each call to ConvertTable.
 	 */
 	MemoryContext convertTableContext =
 		AllocSetContextCreate(CurrentMemoryContext,
 							  "citus_convert_table_context",
 							  ALLOCSET_DEFAULT_SIZES);
 	MemoryContext oldContext = MemoryContextSwitchTo(convertTableContext);
 	TableConversionReturn *tableConversionReturn = ConvertTableInternal(con);
 	MemoryContextSwitchTo(oldContext);
 	/* persist TableConversionReturn in oldContext */
 	TableConversionReturn *tableConversionReturnCopy =
 		CopyTableConversionReturnIntoCurrentContext(tableConversionReturn);
 	/* delete convertTableContext */
 	MemoryContextDelete(convertTableContext);
 	return tableConversionReturnCopy;
 }
 /*
 * DropIndexesNotSupportedByColumnar is a helper function used during accces
 * method conversion to drop the indexes that are not supported by columnarAM.
@ -1129,7 +887,7 @@ DropIndexesNotSupportedByColumnar(Oid relationId, bool suppressNoticeMessages)
 	RelationClose(columnarRelation);
 	Oid indexId = InvalidOid;
-	foreach_declared_oid(indexId, indexIdList)
+	foreach_oid(indexId, indexIdList)
 	{
 		char *indexAmName = GetIndexAccessMethodName(indexId);
 		if (extern_ColumnarSupportsIndexAM(indexAmName))
@ -1310,25 +1068,6 @@ EnsureTableNotPartition(Oid relationId)
 }
 /*
 * ErrorIfColocateWithTenantTable errors out if given colocateWith text refers to
 * a tenant table.
 */
 void
 ErrorIfColocateWithTenantTable(char *colocateWith)
 {
 	if (colocateWith != NULL &&
 		!IsColocateWithDefault(colocateWith) &&
 		!IsColocateWithNone(colocateWith))
 	{
 		text *colocateWithTableNameText = cstring_to_text(colocateWith);
 		Oid colocateWithTableId = ResolveRelationId(colocateWithTableNameText, false);
 		ErrorIfTenantTable(colocateWithTableId,
 						   TenantOperationNames[TENANT_COLOCATE_WITH]);
 	}
 }
 TableConversionState *
 CreateTableConversion(TableConversionParameters *params)
 {
@ -1352,7 +1091,19 @@ CreateTableConversion(TableConversionParameters *params)
 							   "because no such table exists")));
 	}
-
+	TupleDesc relationDesc = RelationGetDescr(relation);
 	if (RelationUsesIdentityColumns(relationDesc))
 	{
 		/*
 		 * pg_get_tableschemadef_string doesn't know how to deparse identity
 		 * columns so we cannot reflect those columns when creating table
 		 * from scratch. For this reason, error out here.
 		 */
 		ereport(ERROR, (errmsg("cannot complete command because relation "
 							   "%s has identity column",
 							   generate_qualified_relation_name(con->relationId)),
 						errhint("Drop the identity columns and re-try the command")));
 	}
 	relation_close(relation, NoLock);
 	con->distributionKey =
 		BuildDistributionKeyFromColumnName(con->relationId, con->distributionColumn,
@ -1389,7 +1140,7 @@ CreateTableConversion(TableConversionParameters *params)
 		 * since they will be handled separately.
 		 */
 		Oid colocatedTableId = InvalidOid;
-		foreach_declared_oid(colocatedTableId, colocatedTableList)
+		foreach_oid(colocatedTableId, colocatedTableList)
 		{
 			if (PartitionTable(colocatedTableId))
 			{
@ -1472,15 +1223,8 @@ CreateDistributedTableLike(TableConversionState *con)
 		newShardCount = con->shardCount;
 	}
 	/*
 	 * To get the correct column name, we use the original relation id, not the
 	 * new relation id. The reason is that the cached attributes of the original
 	 * and newly created tables are not the same if the original table has
 	 * dropped columns (dropped columns are still present in the attribute cache)
 	 * Detailed example in https://github.com/citusdata/citus/pull/6387
 	 */
 	char *distributionColumnName =
-		ColumnToColumnName(con->relationId, (Node *) newDistributionKey);
+		ColumnToColumnName(con->newRelationId, (Node *) newDistributionKey);
 	Oid originalRelationId = con->relationId;
 	if (con->originalDistributionKey != NULL && PartitionTable(originalRelationId))
@ -1504,7 +1248,7 @@ CreateDistributedTableLike(TableConversionState *con)
 	char partitionMethod = PartitionMethod(con->relationId);
 	CreateDistributedTable(con->newRelationId, distributionColumnName, partitionMethod,
-						   newShardCount, true, newColocateWith);
+						   newShardCount, true, newColocateWith, false);
 }
@ -1516,24 +1260,13 @@ void
 CreateCitusTableLike(TableConversionState *con)
 {
 	if (IsCitusTableType(con->relationId, DISTRIBUTED_TABLE))
 	{
 		if (IsCitusTableType(con->relationId, SINGLE_SHARD_DISTRIBUTED))
 		{
 			ColocationParam colocationParam = {
 				.colocationParamType = COLOCATE_WITH_TABLE_LIKE_OPT,
 				.colocateWithTableName = quote_qualified_identifier(con->schemaName,
 																	con->relationName)
 			};
 			CreateSingleShardTable(con->newRelationId, colocationParam);
 		}
 		else
 	{
 		CreateDistributedTableLike(con);
 	}
 	}
 	else if (IsCitusTableType(con->relationId, REFERENCE_TABLE))
 	{
-		CreateReferenceTable(con->newRelationId);
+		CreateDistributedTable(con->newRelationId, NULL, DISTRIBUTE_BY_NONE, 0, false,
 							   NULL, false);
 	}
 	else if (IsCitusTableType(con->relationId, CITUS_LOCAL_TABLE))
 	{
@ -1605,7 +1338,7 @@ DoesCascadeDropUnsupportedObject(Oid classId, Oid objectId, HTAB *nodeMap)
 																	 targetObjectId);
 	HeapTuple depTup = NULL;
-	foreach_declared_ptr(depTup, dependencyTupleList)
+	foreach_ptr(depTup, dependencyTupleList)
 	{
 		Form_pg_depend pg_depend = (Form_pg_depend) GETSTRUCT(depTup);
@ -1645,7 +1378,7 @@ GetViewCreationCommandsOfTable(Oid relationId)
 	List *commands = NIL;
 	Oid viewOid = InvalidOid;
-	foreach_declared_oid(viewOid, views)
+	foreach_oid(viewOid, views)
 	{
 		StringInfo query = makeStringInfo();
@ -1674,16 +1407,17 @@ GetViewCreationCommandsOfTable(Oid relationId)
 /*
- * WrapTableDDLCommands takes a list of command strings and wraps them
+ * GetViewCreationTableDDLCommandsOfTable is the same as GetViewCreationCommandsOfTable,
- * in TableDDLCommand structs.
+ * but the returned list includes objects of TableDDLCommand's, not strings.
 */
-static List *
+List *
-WrapTableDDLCommands(List *commandStrings)
+GetViewCreationTableDDLCommandsOfTable(Oid relationId)
 {
 	List *commands = GetViewCreationCommandsOfTable(relationId);
 	List *tableDDLCommands = NIL;
 	char *command = NULL;
-	foreach_declared_ptr(command, commandStrings)
+	foreach_ptr(command, commands)
 	{
 		tableDDLCommands = lappend(tableDDLCommands, makeTableDDLCommandString(command));
 	}
@ -1745,7 +1479,9 @@ CreateMaterializedViewDDLCommand(Oid matViewOid)
 {
 	StringInfo query = makeStringInfo();
-	char *qualifiedViewName = generate_qualified_relation_name(matViewOid);
+	char *viewName = get_rel_name(matViewOid);
 	char *schemaName = get_namespace_name(get_rel_namespace(matViewOid));
 	char *qualifiedViewName = quote_qualified_identifier(schemaName, viewName);
 	/* here we need to get the access method of the view to recreate it */
 	char *accessMethodName = GetAccessMethodForMatViewIfExists(matViewOid);
@ -1761,7 +1497,10 @@ CreateMaterializedViewDDLCommand(Oid matViewOid)
 	 * Set search_path to NIL so that all objects outside of pg_catalog will be
 	 * schema-prefixed.
 	 */
-	int saveNestLevel = PushEmptySearchPath();
+	OverrideSearchPath *overridePath = GetOverrideSearchPath(CurrentMemoryContext);
 	overridePath->schemas = NIL;
 	overridePath->addCatalog = true;
 	PushOverrideSearchPath(overridePath);
 	/*
 	 * Push the transaction snapshot to be able to get vief definition with pg_get_viewdef
@ -1773,7 +1512,7 @@ CreateMaterializedViewDDLCommand(Oid matViewOid)
 	char *viewDefinition = TextDatumGetCString(viewDefinitionDatum);
 	PopActiveSnapshot();
-	PopEmptySearchPath(saveNestLevel);
+	PopOverrideSearchPath();
 	appendStringInfo(query, "AS %s", viewDefinition);
@ -1794,8 +1533,9 @@ ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommands,
 			 bool suppressNoticeMessages)
 {
 	char *sourceName = get_rel_name(sourceId);
-	char *qualifiedSourceName = generate_qualified_relation_name(sourceId);
+	char *targetName = get_rel_name(targetId);
-	char *qualifiedTargetName = generate_qualified_relation_name(targetId);
+	Oid schemaId = get_rel_namespace(sourceId);
 	char *schemaName = get_namespace_name(schemaId);
 	StringInfo query = makeStringInfo();
@ -1803,7 +1543,8 @@ ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommands,
 	{
 		if (!suppressNoticeMessages)
 		{
-			ereport(NOTICE, (errmsg("moving the data of %s", qualifiedSourceName)));
+			ereport(NOTICE, (errmsg("moving the data of %s",
 									quote_qualified_identifier(schemaName, sourceName))));
 		}
 		if (!HasAnyGeneratedStoredColumns(sourceId))
@ -1813,7 +1554,8 @@ ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommands,
 			 * "INSERT INTO .. SELECT *"".
 			 */
 			appendStringInfo(query, "INSERT INTO %s SELECT * FROM %s",
-							 qualifiedTargetName, qualifiedSourceName);
+							 quote_qualified_identifier(schemaName, targetName),
 							 quote_qualified_identifier(schemaName, sourceName));
 		}
 		else
 		{
@ -1826,21 +1568,18 @@ ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommands,
 			 */
 			List *nonStoredColumnNameList = GetNonGeneratedStoredColumnNameList(sourceId);
 			char *insertColumnString = StringJoin(nonStoredColumnNameList, ',');
-			appendStringInfo(query,
+			appendStringInfo(query, "INSERT INTO %s (%s) SELECT %s FROM %s",
-							 "INSERT INTO %s (%s) OVERRIDING SYSTEM VALUE SELECT %s FROM %s",
+							 quote_qualified_identifier(schemaName, targetName),
-							 qualifiedTargetName, insertColumnString,
+							 insertColumnString, insertColumnString,
-							 insertColumnString, qualifiedSourceName);
+							 quote_qualified_identifier(schemaName, sourceName));
 		}
 		ExecuteQueryViaSPI(query->data, SPI_OK_INSERT);
 	}
-	/*
+	List *ownedSequences = getOwnedSequences(sourceId);
 	 * Modify regular sequence dependencies (sequences marked as DEPENDENCY_AUTO)
 	 */
 	List *ownedSequences = getOwnedSequences_internal(sourceId, 0, DEPENDENCY_AUTO);
 	Oid sequenceOid = InvalidOid;
-	foreach_declared_oid(sequenceOid, ownedSequences)
+	foreach_oid(sequenceOid, ownedSequences)
 	{
 		changeDependencyFor(RelationRelationId, sequenceOid,
 							RelationRelationId, sourceId, targetId);
@ -1853,50 +1592,62 @@ ReplaceTable(Oid sourceId, Oid targetId, List *justBeforeDropCommands,
 		 */
 		if (ShouldSyncTableMetadata(targetId))
 		{
-			char *qualifiedSequenceName = generate_qualified_relation_name(sequenceOid);
+			Oid sequenceSchemaOid = get_rel_namespace(sequenceOid);
 			char *sequenceSchemaName = get_namespace_name(sequenceSchemaOid);
 			char *sequenceName = get_rel_name(sequenceOid);
 			char *workerChangeSequenceDependencyCommand =
-				CreateWorkerChangeSequenceDependencyCommand(qualifiedSequenceName,
+				CreateWorkerChangeSequenceDependencyCommand(sequenceSchemaName,
-															qualifiedSourceName,
+															sequenceName,
-															qualifiedTargetName);
+															schemaName, sourceName,
 															schemaName, targetName);
 			SendCommandToWorkersWithMetadata(workerChangeSequenceDependencyCommand);
 		}
 		else if (ShouldSyncTableMetadata(sourceId))
 		{
 			char *qualifiedTableName = quote_qualified_identifier(schemaName, sourceName);
 			/*
 			 * We are converting a citus local table to a distributed/reference table,
 			 * so we should prevent dropping the sequence on the table. Otherwise, we'd
 			 * lose track of the previous changes in the sequence.
 			 */
-			char *command = WorkerDropSequenceDependencyCommand(sourceId);
+			StringInfo command = makeStringInfo();
-			SendCommandToWorkersWithMetadata(command);
+
 			appendStringInfo(command,
 							 "SELECT pg_catalog.worker_drop_sequence_dependency(%s);",
 							 quote_literal_cstr(qualifiedTableName));
 			SendCommandToWorkersWithMetadata(command->data);
 		}
 	}
 	char *justBeforeDropCommand = NULL;
-	foreach_declared_ptr(justBeforeDropCommand, justBeforeDropCommands)
+	foreach_ptr(justBeforeDropCommand, justBeforeDropCommands)
 	{
 		ExecuteQueryViaSPI(justBeforeDropCommand, SPI_OK_UTILITY);
 	}
 	if (!suppressNoticeMessages)
 	{
-		ereport(NOTICE, (errmsg("dropping the old %s", qualifiedSourceName)));
+		ereport(NOTICE, (errmsg("dropping the old %s",
 								quote_qualified_identifier(schemaName, sourceName))));
 	}
 	resetStringInfo(query);
 	appendStringInfo(query, "DROP %sTABLE %s CASCADE",
 					 IsForeignTable(sourceId) ? "FOREIGN " : "",
-					 qualifiedSourceName);
+					 quote_qualified_identifier(schemaName, sourceName));
 	ExecuteQueryViaSPI(query->data, SPI_OK_UTILITY);
 	if (!suppressNoticeMessages)
 	{
-		ereport(NOTICE, (errmsg("renaming the new table to %s", qualifiedSourceName)));
+		ereport(NOTICE, (errmsg("renaming the new table to %s",
 								quote_qualified_identifier(schemaName, sourceName))));
 	}
 	resetStringInfo(query);
 	appendStringInfo(query, "ALTER TABLE %s RENAME TO %s",
-					 qualifiedTargetName,
+					 quote_qualified_identifier(schemaName, targetName),
 					 quote_identifier(sourceName));
 	ExecuteQueryViaSPI(query->data, SPI_OK_UTILITY);
 }
@ -1987,7 +1738,7 @@ CheckAlterDistributedTableConversionParameters(TableConversionState *con)
 		Oid colocatedTableOid = InvalidOid;
 		text *colocateWithText = cstring_to_text(con->colocateWith);
 		Oid colocateWithTableOid = ResolveRelationId(colocateWithText, false);
-		foreach_declared_oid(colocatedTableOid, con->colocatedTableList)
+		foreach_oid(colocatedTableOid, con->colocatedTableList)
 		{
 			if (colocateWithTableOid == colocatedTableOid)
 			{
@ -2006,12 +1757,6 @@ CheckAlterDistributedTableConversionParameters(TableConversionState *con)
 								   "it is not a distributed table",
 								   con->colocateWith)));
 		}
 		else if (IsCitusTableType(colocateWithTableOid, SINGLE_SHARD_DISTRIBUTED))
 		{
 			ereport(ERROR, (errmsg("cannot colocate with %s because "
 								   "it is a single shard distributed table",
 								   con->colocateWith)));
 		}
 	}
 	/* shard_count:=0 is not allowed */
@ -2156,13 +1901,18 @@ CheckAlterDistributedTableConversionParameters(TableConversionState *con)
 * worker_change_sequence_dependency query with the parameters.
 */
 static char *
-CreateWorkerChangeSequenceDependencyCommand(char *qualifiedSequeceName,
+CreateWorkerChangeSequenceDependencyCommand(char *sequenceSchemaName, char *sequenceName,
-											char *qualifiedSourceName,
+											char *sourceSchemaName, char *sourceName,
-											char *qualifiedTargetName)
+											char *targetSchemaName, char *targetName)
 {
 	char *qualifiedSchemaName = quote_qualified_identifier(sequenceSchemaName,
 														   sequenceName);
 	char *qualifiedSourceName = quote_qualified_identifier(sourceSchemaName, sourceName);
 	char *qualifiedTargetName = quote_qualified_identifier(targetSchemaName, targetName);
 	StringInfo query = makeStringInfo();
 	appendStringInfo(query, "SELECT worker_change_sequence_dependency(%s, %s, %s)",
-					 quote_literal_cstr(qualifiedSequeceName),
+					 quote_literal_cstr(qualifiedSchemaName),
 					 quote_literal_cstr(qualifiedSourceName),
 					 quote_literal_cstr(qualifiedTargetName));
@ -2214,7 +1964,7 @@ WillRecreateForeignKeyToReferenceTable(Oid relationId,
 	{
 		List *colocatedTableList = ColocatedTableList(relationId);
 		Oid colocatedTableOid = InvalidOid;
-		foreach_declared_oid(colocatedTableOid, colocatedTableList)
+		foreach_oid(colocatedTableOid, colocatedTableList)
 		{
 			if (HasForeignKeyToReferenceTable(colocatedTableOid))
 			{
@ -2242,7 +1992,7 @@ WarningsForDroppingForeignKeysWithDistributedTables(Oid relationId)
 	List *foreignKeys = list_concat(referencingForeingKeys, referencedForeignKeys);
 	Oid foreignKeyOid = InvalidOid;
-	foreach_declared_oid(foreignKeyOid, foreignKeys)
+	foreach_oid(foreignKeyOid, foreignKeys)
 	{
 		ereport(WARNING, (errmsg("foreign key %s will be dropped",
 								 get_constraint_name(foreignKeyOid))));
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Onur Tirtir	9b19e41e46	Add changelog entries for 11.1.2 (#6386 ) (cherry picked from commit `17cf137c4c`)	2022-09-30 13:07:28 +03:00
Onur Tirtir	10be12e4be	Bump citus version to 11.1.2	2022-09-30 13:06:12 +03:00
Jelte Fennema	006f6aceaf	Reuse connections for Splits and Logical Replication (#6314 ) In Split, Logical replication logic and ShardCleaner we call `SendCommandListToWorkerOutsideTransaction` and `SendOptionalCommandListToWorkerOutsideTransaction` frequently. This opens new connection for each of those calls, even though we already have a perfectly good connection lying around. This PR adds two new APIs `SendCommandListToWorkerOutsideTransactionWithConnection` and `SendOptionalCommandListToWorkerOutsideTransactionWithConnection` that allow sending a list of queries in a transaction over an existing connection. We also update the callers (Split, ShardCleaner, Logical Replication) to use these new APIs instead. Co-authored-by: Nitish Upreti <niupre@microsoft.com> Co-authored-by: Onder Kalaci <onderkalaci@gmail.com> (cherry picked from commit `24e06af6d2`)	2022-09-26 16:53:38 +02:00
Jelte Fennema	ebe70adc92	Revert replica identity creation order for shard moves (#6367 ) In Citus 11.1.0 we changed the order of doing the initial data copy and the replica identity creation when doing a non blocking shard move. This was done to try and increase the speed with which shard moves could be done. But after doing more extensive performance testing this change turned out to have a negative impact on the speed of moves on the setups that I tested. Looking at the resource usage metrics of the VMs the reason for this seems to be that these shard moves were bottlenecked by disk bandwidth. While creating replica identities in bulk after the initial copy will reduce CPU usage a bit, it does require an additional sequence scan of the just written data. So when a VM is bottlenecked on disk, it makes sense to spend a little bit more CPU to avoid an additional scan. Since PKs are usually simple indexes that don't require lots of CPU to update, as opposed to e.g. GiST indexes. This reverts the order change to avoid a regression on shard move speed in these cases. For future releases we might consider re-evaluating our index creation order for other indexes too, and create "simple" indexes before the copy. (cherry picked from commit `d9a9a3263b`)	2022-09-26 16:53:30 +02:00
Onur Tirtir	b9e4364acc	Not allow ON DELETE/UPDATE SET DEFAULT actions on columns that default to sequences (#6340 ) Given that we drop DEFAULT nextval('sequence') expressions from shard relation columns, allowing `ON DELETE/UPDATE SET DEFAULT` on such columns might cause inserting NULL values as a result of a delete/update operation. For this reason, we disallow ON DELETE/UPDATE SET DEFAULT actions on columns that default to sequences. DESCRIPTION: Disallows having ON DELETE/UPDATE SET DEFAULT actions on columns that default to sequences Fixes #6339. (cherry picked from commit `a868cc049a`) Conflicts: src/test/regress/expected/pg15.out src/test/regress/sql/pg15.sql	2022-09-23 13:55:51 +03:00
Onur Tirtir	53ec5abb75	Not drop default col exprs from shard when adding local table to metadata (#6323 ) As we did for GENERATED STORED columns in #4613, we should not drop column default expressions that are not based on sequences from shard relation since such expressions need to exist e.g. for foreign key actions. For the column default expressions that are based on sequences we cannot do much, so we need to disallow having ON DELETE SET DEFAULT actions on such columns in a separate PR, see #6339. Fixes #6318. DESCRIPTION: Fixes a bug that might cause inserting incorrect DEFAULT values when applying foreign key actions (cherry picked from commit `de24a3eda5`)	2022-09-23 13:53:04 +03:00
Ahmet Gedemenli	ecaa0cda6d	Fix dropping replication slot (#6359 ) DESCRIPTION: Fixes dropping replication slots As detected by a flaky test, Citus sometimes fails to drop replication slots, possibly due to a race condition, at the end of a shard split. With this PR, we retry to drop them in case of an `OBJECT_IN_USE` error, consistently for 20 seconds. fixes: #6326 (cherry picked from commit `bae4b47c2f`)	2022-09-22 11:31:37 +03:00
Nitish Upreti	a8e7c2cb09	Shard Split : Add / Update logging (#6336 ) DESCRIPTION: Improve logging during shard split and resource cleanup ### DESCRIPTION This PR makes logging improvements to Shard Split : 1. Update confusing logging to fix #6312 2. Added new `ereport(LOG` to make debugging easier as part of telemetry review.	2022-09-16 10:11:37 -07:00
Onur Tirtir	b7ae596fe8	Bump citus version to 11.1.1 (#6356 )	2022-09-16 12:33:09 +03:00
Onur Tirtir	6f4324623c	Add changelog entries for 11.1.1 (#6354 ) (cherry picked from commit `8b5cdaf0e9`)	2022-09-16 12:17:25 +03:00
Marco Slot	d5db0adc17	Allow create_distributed_table_concurrently on an empty node (#6353 ) Co-authored-by: Marco Slot <marco.slot@gmail.com>	2022-09-16 12:17:25 +03:00
Onur Tirtir	099523452e	Add changelog entries for 11.1.0 (#6349 ) Created by executing `prepare_changelog.pl citus 11.1.0 2022-03-29`. (cherry picked from commit `57e354ac91`)	2022-09-16 11:20:00 +03:00
Onder Kalaci	af448da1a7	Prevent failures on partitioned distributed tables with statistics objects on PG 15 Comment from the code is clear on this: /* * The statistics objects of the distributed table are not relevant * for the distributed planning, so we can override it. * * Normally, we should not need this. However, the combination of * Postgres commit 269b532aef55a579ae02a3e8e8df14101570dfd9 and * Citus function AdjustPartitioningForDistributedPlanning() * forces us to do this. The commit expects statistics objects * of partitions to have "inh" flag set properly. Whereas, the * function overrides "inh" flag. To avoid Postgres to throw error, * we override statlist such that Postgres does not try to process * any statistics objects during the standard_planner() on the * coordinator. In the end, we do not need the standard_planner() * on the coordinator to generate an optimized plan. We call * into standard_planner() for other purposes, such as generating the * relationRestrictionContext here. * * AdjustPartitioningForDistributedPlanning() is a hack that we use * to prevent Postgres' standard_planner() to expand all the partitions * for the distributed planning when a distributed partitioned table * is queried. It is required for both correctness and performance * reasons. Although we can eliminate the use of the function for * the correctness (e.g., make sure that rest of the planner can handle * partitions), it's performance implication is hard to avoid. Certain * planning logic of Citus (such as router or query pushdown) relies * heavily on the relationRestrictionList. If * AdjustPartitioningForDistributedPlanning() is removed, all the * partitions show up in the, causing high planning times for * such queries. */	2022-09-15 14:37:28 +03:00
Sameer Awasekar	acccad9879	Introduce code changes to fix Issue:6303 (#6328 ) The PR introduces code changes to fix Issue [6303](https://github.com/citusdata/citus/issues/6303) `create_distributed_table_concurrently` following drop column, creates a buggy situation in split decoder. * Consider the below scenario: * Session1 : Drop column followed by create_distributed_table_concurrently * Session2 : Concurrent insert workload The child shards created by `create_distributed_table_concurrently` will have less columns than the source shard because some column were dropped. The incoming tuple from session2 will have more columns as the writes happened on source shard. But now the tuple needs to be applied on child shard. So we need to format existing tuple according to child schema and skip dropped column values. The PR fixes this by reformatting the tuple according the target child schema. Test: 1) isolation_create_distributed_concurrently_after_drop_column - Repros the issue and tests on the same.	2022-09-15 09:48:43 +02:00
aykut-bozkurt	77947da17c	ensure we have more active nodes than replication factor. (#6341 ) DESCRIPTION: Fixes floating exception during create_distributed_table_concurrently. Fixes #6332. During create_distributed_table_concurrently, when there is no active primary node, it fails with floating exception. We added similar check with create_distributed_table. It will fail with proper message if current active node is less than replication factor. (cherry picked from commit `739b91afa6`)	2022-09-14 18:22:58 +03:00
Marco Slot	7d56c25e28	Fix escaping in sequence dependency queries (#6345 ) Co-authored-by: Marco Slot <marco.slot@gmail.com>	2022-09-14 16:52:25 +02:00
Marco Slot	eba70af7a2	Fix bugs in CheckIfRelationWithSameNameExists (#6343 ) Co-authored-by: Marco Slot <marco.slot@gmail.com>	2022-09-14 15:59:43 +02:00
Hanefi Onaldi	3f33390f45	Bump Citus version to 11.1.0	2022-09-14 14:21:43 +03:00
Nils Dijk	7b51f3eee2	Fix: rebalance stop non super user (#6334 ) No need for description, fixing issue introduced with new feature for 11.1 Fixes #6333 Due to Postgres' C api being o-indexed and postgres' attributes being 1-indexed, we were reading the wrong Datum as the Task owner when cancelling. Here we add a test to show the error and fix the off-by-one error.	2022-09-13 23:20:06 +02:00