command line - Giving cat output to rm

I have file names in a file which I need to delete in a different directory.

Let's say I have x and y files in dir a. How do I delete it using cat?

I tried,

rm -f a/{`cat a.txt`}

a.txt has contents x,y,z.

If they are in the same folder, I can put x y z in a.txt and run,

rm -f `cat a.txt`

which works fine.

I have also tried,

rm -f "a/{"`cat a.txt`"}"

This command will go in a dockerfile so I prefer not to use any variables too.

I do not want to put a/x a/y a/z in the file which can be an option, as it is fixed that a will only contain the files. But a should be changed only in the dockerfile. Thanks for all suggestions in advance :)

Assuming your filenames don't contain spaces or any special characters, just reuse your original command with a cd before it:

(cd a; rm -f $(cat a.txt))

Be warned that rm -f `cat a.txt` breaks easily with spaces or any special characters in filenames, you should really use xargs with NUL-delimited filenames.