0

My code goes something like this:

echo "$result" | xargs -P50 -I@ bash -c '{ printf "@ $(curl --write-out "%{http_code}" -L -s --output /dev/null @)\n"; }'

Where result is containing a list similar to :

https://example.com/somepath
https://example.com/somepath&quot
https://example.com/anotherpath?par1&par4=gg

The expected output [URL STATUS-CODE]

https://example.com/somepath 200
https://example.com/somepath&quot 200
https://example.com/anotherpath?par1&par4=gg 200

The issue is when running this snippet, i get the error bash: quot: No such file or directory which is separated by the andsign & and i cant seem to work around it

I tried wrapping @ in single/double quotes but nothing worked

Raywando
  • 497
  • 1
    Please [edit] your question and show how you set the value of result. Maybe it would be easier without putting the multi-line data in a single variable and splitting it using xargs. Avoiding subshells and using printf with a proper format string and modified quoting may also help. Explain in your question what you want to achieve. Based on guessing from the code and example data you somehow get a list of URLs and want to display the reply code of the web server when you try to access the URL using curl. Do you want to further process the reply code later? – Bodo Mar 08 '21 at 12:29
  • Use printf correctly. printf FORMAT [ARGUMENT] like @Bodo said, then check this for safe usage of bash -c (the question is about find -exec sh -c , but the very same applies here. Also, you might get rid of the superfluous { and }. – pLumo Mar 08 '21 at 12:36
  • @Bodo The result variable is set in different ways in the code, I showed how the final value would be, and yes you are right, i want to get the status code of each URL and i will need both urls along with the status code for later in the code. I shared a sample of result variable and the expected output. – Raywando Mar 08 '21 at 12:38
  • @Raywando Explaining what is already clear from the question does not help much. Please [edit] your question to add clarification or requested information. Don't use comments for this purpose. "The result variable is set in different ways" doesn't clarify anything. An example why it may be useful to know how you construct the result value: If you generate the URLs in a loop it might be easier to call curl in the same loop. When you generate multi-line output where every line is one value, it might be better to feed this into a while IFS= read -r variable loop. – Bodo Mar 08 '21 at 12:47

1 Answers1

1

Embedding the replacement in command strings introduces command injection vulnerabilities. Which is what you are seeing in action: instead of being (just) used as a string argument to curl, the content of result is being parsed as code.

To prevent that, you may use (focusing on the main issue only; other improvements are possible):

printf '%s\n' "$result" | xargs -P50 -I@ bash -c 'printf "%s %s\n" "$1" \
  "$(curl --write-out "%{http_code}" -L -s --output /dev/null "$1")"' mybash @
fra-san
  • 10,205
  • 2
  • 22
  • 43
  • 1
    Maybe nitpicking, but as http_code will be always a digit, you should use %d for it. You could also add $1 to write-out and completely omit printf, but not sure about the safety implications of it .... – pLumo Mar 08 '21 at 12:42
  • Thanks! @fra-san, but whats mybash here? – Raywando Mar 08 '21 at 12:43
  • 1
    @Raywando mybash is an arbitrary string used as the command name when a shell is invoked with the -c option. See the description of -c in the Bash manual. – fra-san Mar 08 '21 at 12:52
  • @fra-san I suggest to add the explanation about mybash to the answer. – Bodo Mar 08 '21 at 12:57
  • maybe you could do --write-out "%{url_effective} %{http_code}\n" and skip that second printf altogether. (maybe even the bash -c command_string too) –  Mar 08 '21 at 13:01
  • @pLumo Not sure about --write-out, I'd leave it to others. Also not sure whether %d would be better, given that for the shell it is just text anyway and there seems to be no need for number formatting here (alignment, padding, ...). Please feel free to directly edit my answer if you see room for improvement. – fra-san Mar 08 '21 at 13:07
  • xargs <<< "$result" -n 1 curl --write-out "%{url_effective} %{http_code}\n" -L -s --output /dev/null –  Mar 08 '21 at 13:07
  • 1
    That is what I thought too @bac0n, but url_effective is maybe not what OP wants, it may differ from the input. – pLumo Mar 08 '21 at 13:15
  • or ... -I @ curl --write-out "@ %{http_code}\n" ... @ –  Mar 08 '21 at 13:19