external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Use rename to make sure no files are inserted while deleting (#3912) 

As suggested by @marcocitus in https://github.com/citusdata/citus/pull/3911#issuecomment-643978531, there was
a regression in #3893. If another backend would write a file during deletion of
the intermediate results directory, this file would not necessarily be deleted.

The approach used in `CitusRemoveDirectory` is to try recursive removal of the
directory again if it has failed. This does not work here, since when a file
can not be removed for other reasons (e.g. `EPERM`) it will not throw an error
anymore. So then we would get into an infinite removal loop. Instead I now
`rename` the directory before removing it. That way other backends will not
write files to it anymore.
pull/3872/head^2
Jelte Fennema 2020-06-23 10:38:44 +02:00 committed by GitHub
parent 0e0695481c
commit a98226842d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 27 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
src/backend/distributed/executor/intermediate_results.c
View File

@ -701,7 +701,33 @@ RemoveIntermediateResultsDirectory(void)
{
if (CreatedResultsDirectory)
{
PathNameDeleteTemporaryDir(IntermediateResultsDirectory());
/*
* The shared directory is renamed before deleting it. Otherwise it
* would be possible for another backend to write a file, while we are
* deleting the directory. Since rename is atomic by POSIX standards
* that's not possible. The current PID is included in the new
* filename, so there can be no collisions with other backends.
*/
char *sharedName = IntermediateResultsDirectory();
StringInfo privateName = makeStringInfo();
appendStringInfo(privateName, "%s.removed-by-%d", sharedName, MyProcPid);
if (rename(sharedName, privateName->data))
{
ereport(LOG,
(errcode_for_file_access(),
errmsg(
"could not rename intermediate results directory \"%s\" to \"%s\": %m",
sharedName, privateName->data)));
/* rename failed for some reason, we do a best effort removal of
* the shared directory */
PathNameDeleteTemporaryDir(sharedName);
}
else
{
PathNameDeleteTemporaryDir(privateName->data);
}
CreatedResultsDirectory = false;
}