1

HELP!!! Trying to split a csv file admin_bids_view.csv into multiple files. The CSV has 2000 rows and 7 columns. I would like a new file per ID number in column 1.

Example data below:

enter image description here

Have tried awk -F\| '{print>$1}' admin_bids_view.csv

But the below is being returned.

awk: can't open file admin_bids_view.csv source line number 1

Please can someone point me in the right direction?

Thank you.

SJB
  • 13

1 Answers1

4

Your approach would mostly work, although you'd end up with column A in the output file, which may not be what you want. Here's another approach that doesn't put column A in the output file:

awk -F, '{outfile=($1 ".csv") ; print substr($0,index($0,$2)) >>outfile ; close(outfile)}' <name_of_input_file

You said "CSV", so use a comma as the field separator.

Marc Wilson
  • 5,920
  • Thank you. That is exactly what I need it to do. I got the following error: awk -F, "{print substr($0,index($0,$2)) >$1}" admin_bids_view.csv awk: syntax error at source line 1 context is

    {print(-bash,index(-bash,) <<<

    awk: illegal statement at source line 1 awk: illegal statement at source line 1

     – SJB Nov 22 '18 at 16:57
  • Hm. Replace the double quotes with single quotes and try it. I tested my answer, but on a Windows machine. ^_^ – Marc Wilson Nov 22 '18 at 17:25
  • That worked, thank you. The next problem encountered is: awk: 263 makes too many open files Is there an easy solution to this. Sorry for all the questions. – SJB Nov 22 '18 at 18:29
  • Yeah, your problem is actually two related problems. You need awk to keep each file it creates open so that it can append to it, and keeping the file open eventually exhausts the available file descriptors (there must be a LARGE number of unique values in column A). I've edited my original answer... try the edit and see if it works. Bear in mind that since this is now using an append, you need to erase the result files before you run it. – Marc Wilson Nov 22 '18 at 19:42
  • You are an absolute lifesaver, that worked perfectly. Final question, I promise, is there a way to add .csv to the file names? Thank you. – SJB Nov 22 '18 at 20:14
  • Ok, final edit. If that works, please accept the answer. – Marc Wilson Nov 22 '18 at 20:37
  • Being an old Unix dude, I want to suggest cut(1), which is a tad simpler. But Marc's answer is the right one because (1) awk is worth learning if you're going to write shell scripts and (2) cut works differently on different OSs. – Isaac Rabinovitch Nov 22 '18 at 21:31