I am on Windows, collecting a dataset and have to repeatedly perform following operations.
I'm looking for a way to automate some of them.
General description
- Run a process that endlessly prints its data to stdout
- Redirect that stdout to a file using
> - Wait until the file is longer than 1200 lines (open the file in a viewer and watch for changes)
- Ctrl-C
- Reconfigure the process and go to 1
I am looking for some kind of pipe filter that would do steps 3 and 4 for me. I can use either cmd.exe or Powershell.
Details
I am collecting data from a ROS2 node. Here are specific commands and my actions.
Steps 1, 2: run the node, then open a separate ROS2 console and issue the command ros2 topic echo --csv --field range /range > filename.txt.
Step 3: open the file filename.txt (it is immediately created) in gvim and issue commands :set autoread | au CursorHold * checktime | call feedkeys("lh"). These commands make vim to watch for changes in the file and reload it from the disk automatically. More on them is here.
After some time of waiting filename.txt contains the following:
...
0.7152443528175354
0.7513738870620728
0.725148618221283
0.7509835362434387
0.7565790414810181
0.7424352765083313
0.7426444292068481
0.7367836833000183
0.7558256387710571
0.7744500637054443
0.7122563123703003
0.7287737727165222
0.780799388885498
...
Since getting chunk of 1200 lines takes about 10-12 min, I run the command and switch to another task. The console window goes to the background, and now I need some way to get notification that required amount of data is collected.
Vim shows me the number of lines in the file. Currently I keep its window open on a second monitor and periodically glance at that number while doing other things. When I notice that more than 1200 lines are collected, I switch to the ROS2 console and press Ctrl-C.
Then I stop my ROS2 node, reconfigure it and run again.
I am looking for some filter so that I could issue the following command
ros2 topic echo --csv --field range /range | THAT-FILTER --1200 > filename.txt
I expect from that filter to break the pipe and notify me somehow that it has collected required amount of data.
Things that I've tried and that didn't work.
I've looked at tqdm, but it doesn't seem to have the option to terminate the pipe.
Following suggestions of Kamil Maciorowski in comments I've tried to use
headandselect -first nfrom Powershell. No luck.This command never ends:
ros2 topic echo ... | "C:\Program Files\Git\usr\bin\head.exe" -n 1200 > file.txtAlthough
file.txtcontains exactly 1200 lines.Similarly to
head.exe,select -first 10prints first 10 lines but doesn't break the pipe. And one more issue: encoding of created file is U16Le, not ANSI.I've found that notifications can be implemented using notify-send for Windows. However, I need pipe termination.
Ultimately, the goal can be achieved with writing one more ROS2 node, that would subscribe to the topic, dump requested amount of data to the file and issue notifications. I wonder if the goal can be achieved with generic OS tools.
head -n 1200would be a solution. Maybe this answer will help you. – Kamil Maciorowski Nov 26 '21 at 09:10