The dxfuse GitHub page mentions a -limitedWrite flag that "allows writing data in append-only fashion" but my group hasn't managed to get this to work on RAP.
We succeeded in running dxfuse -limitedWrite on a worker (running an interactive shell) and found that we could create files and directories within /mnt/project (that then appear in RAP storage). However, the files were still read-only, such that we could not actually write anything to them.
Is this behavior expected? Is the append feature of -limitedWrite somehow restricted to spark file output?
I do not know what is the exact command that you are running. If you are trying something along the lines of creating a file, then appending to it with another process that won't work, it needs to be done by the same process.
e.g. the following doesn't work since dxfuse has already created an empty file on the platform and closed it such that it is immutable.
touch file
echo "my data" >> file
bash: echo: write error: Operation not permitted
However the following works as expected.
echo "my data" >> file1?
0
Permanently deleted user
Thanks! I can't remember the exact experiment we tried, but I tried the way you suggested and indeed it works.
Comments
2 comments
I do not know what is the exact command that you are running. If you are trying something along the lines of creating a file, then appending to it with another process that won't work, it needs to be done by the same process.
e.g. the following doesn't work since dxfuse has already created an empty file on the platform and closed it such that it is immutable.
touch file
echo "my data" >> file
bash: echo: write error: Operation not permitted
However the following works as expected.
echo "my data" >> file1?
Thanks! I can't remember the exact experiment we tried, but I tried the way you suggested and indeed it works.
Please sign in to leave a comment.