Our repo was complaining about the cryptography package being
vulnerable. This updates it, including our mitmproxy fork, because that
was pinning an outdated version.
Relevant commit on our mitmproxy fork:
2fd18ef051
Relevant PR on the-process:
https://github.com/citusdata/the-process/pull/112
There is a vulnerability in mitmproxy with the version we are using.
It would be hard to exploit anything with regards to the artifacts we ship as its only used in our test suite. Still its good hygiene to _not_ use software with known vulnerabilities.
This PR updates the version of python, mitmproxy and the crypto libraries used.
The latest version of mitmproxy for python 3.6 is not patched, hence the upgrade of python.
For our CI images this cascades into upgrading debian as well :)
For CI we bake these versions in our images so we need to update them as well.
Changes to the CI images: https://github.com/citusdata/the-process/pull/65
* Update failure test dependencies
There was a security alert for cryptography. The vulnerability was fixed
in 3.2.0. The vulnebarility:
"RSA decryption was vulnerable to Bleichenbacher timing vulnerabilities,
which would impact people using RSA decryption in online scenarios."
The fix:
58494b41d6
It wasn't enough to only update crpytography because mitm was
incompatible with the new version, so mitm is also upgraded.
The steps to do in local:
python -m pip install -U cryptography
python -m pip install -U mitmproxy
* Add creating a citus cluster script
Creating a citus cluster is automated.
Before running this script:
- Citus should be installed and its control file should be added to postgres. (make install)
- Postgres should be installed.
* Initialize upgrade test table and fill
* Finalize the layout of upgrade tests
Postgres upgrade function is added.
The newly added UDFs(citus_prepare_pg_upgrade, citus_finish_pg_upgrade) are used to
perform upgrade.
* Refactor upgrade test and add config file
* Add schedules for upgrade testing
* Use pg_regress for upgrade tests
pg_regress is used for creating a simple distributed table in
upgrade tests. After upgrading another schedule is used to verify
that the distributed table exists. Router and realtime queries are
used for verifying.
* Run upgrade tests as a postgres user in a temp dir
postgres user is used for psql to be consistent at running tests.
A temp dir is created and the temp dir's permissions are changed so
that postgres user can access it. All psql commands are now run with
postgres user.
"Select * from t" query is changed as "Select * from t order by a"
so that the result is always in the same order.
* Add docopt and arguments for the upgrade script
Docopt dependency is added to parse flags in script.
Some refactoring in variable names is done.
* Add readme for upgrade tests
* Refactor upgrade tests
Use relative data path instead of absolute assuming that this script will
always be run from 'src/test/regress'
Remove 'citus-path' flag
Use specific version for docopt instead of *
Use named args in string formatting
* Resolve a security problem
Instead of using string formatting in subprocess.call, arguments
list is used. Otherwise users could do shell injection.
Shell = True is removed from subprocess call as it is not recommended
to use this.
* Add how the test works to readme
* Refactor some variables to be consistent
* Update upgrade script based on the reviews
It was possible that postgres server would stay running even when the script
crashes, atexit library is used to ensure that we always do a teardown where we stop
the databases.
Some formatting is done in the code for better readability.
Config class is used instead of a dictonary.
A target for upgrade test is added to makefile.
Unused flags/functions/variables are removed.
* Format commands and remove unnecessary flag from readme
- Lots of detail is in src/test/regress/mitmscripts/README
- Create a new target, make check-failure, which runs tests
- Tells travis how to install everything and run the tests
Jelte Fennema
Jelte Fennema
Nils Dijk
SaitTalhaNisanci
Brian Cloutier