2

Kusalananda nicely recommends using csvformat from csvkit to format jq @csv into a csv format without double quotes " answering how to parse json with jq.

This answer does not seem to involve the use of python. But the csvkit installation tutorial and its installation troubleshooting do seem to rely on, perhaps require, the use of python. This makes me, a newbie, confused:

Is it possible to install csvkit in git bash without using python (read: open spyder or anaconda, let's say)? How?

Edit. MINGW64 (git bash) displays bash: pip: command not found. Same for conda. How do you recommend moving on from there?

python is installed, pip.exe being in ...\Anaconda\Scripts. There are several suggested solutions on other sites e.g. in various ways adding the dir of pip.exe to PATH here and here).

Johan
  • 399

2 Answers2

6

The csvkit tools are all written in Python.

Depending on what Unix you're running, you may also find csvkit available as a ready-made package, so you don't have to worry about explicitly dealing with pip or Python packages. On Debian-based Linuxes, the package is called csvkit (so use apt install csvkit). The Homebrew package manager for macOS also has a csvkit package (so use brew install csvkit).

Installing csvkit as a ready-made package from your ordinary software distributor is the easiest way to install csvkit. This installs the software and its dependencies as an integrated part of your system and will keep it up to date as any other software that you have installed in the same way.

However, it's unlikely that the developers of csvkit are the ones who also package the software for your system (it's done by package maintainers associated with the Unix or Linux you are using), so their only surefire way to recommend the installation of the utilities is using pip, the Python package manager.

Using pip to install csvkit as described in the documents that you link to is only neccesary if your ordinary package manager does not have csvkit (as on e.g. OpenBSD), if you need a particular version of csvkit that is not provided in any other way, or if you need to install the software as an unprivileged user (use pip with its --user option).

Care should be taken when installing software by other means than through the use of the ordinary package manager to avoid collisions between installed files. It's, therefore, a good idea to set up a virtualenv environment and install csvkit in there, as suggested by the csvkit developers.

Kusalananda
  • 333,661
  • 1
    +1. Also worth noting: if a packaged version is available for your distro then install the packaged version - it will have all the right dependencies to be properly integrated into the OS, and upgrades will be automatically handled. Installing stuff with pip will require manual upgrading and dependency management in future. It's nice to have tools like pip available, but distro packages are inherently better. Also, some distros have tools to automate creation of distro packages from python packages - e.g. debian's py2dsc (from python3-stdeb`) – cas Jun 20 '22 at 23:57
  • 1
    Some people might not want to use 260 MB just to have pip installed and apt install csvkit is a good option, however, by default, it is still a 190 MB installation, so maybe simply running apt install python3 python3-setuptools and installing manually the latest version (1.0.7) of csvkit is a better option (python3 setup.py install). Bonus points if you create a virtual environment for your python stuff, just to prevent potential conflicts later on (which are not precisely uncommon when installing distro packages). – r_31415 Jun 21 '22 at 02:43
  • @JohanAndresen If you already have Anaconda installed, simply run conda install csvkit. I used Anaconda (actually, miniconda) because I work with python files regularly, however, that wouldn't be my first suggestion for people that only want to use a single python package given the huge number of libraries that anaconda preinstalls by default. – r_31415 Jun 21 '22 at 19:16
  • And yes, you can update your question to clarify the issue, although most people won't receive notifications of the changes unless they are following your question. – r_31415 Jun 21 '22 at 19:17
  • Thanks all. Sorry @RobertSmith, you were so fast I did not get to edit my comment: MINGW64 displays bash: pip: command not found. Same for conda. How do you recommend moving on from there? python is installed, pip.exe being in ...\Anaconda\Scripts. There are several suggested solutions on other sites e.g. in various ways adding the dir of pip.exe to PATH here and here. – Johan Jun 21 '22 at 19:44
  • @JohanAndresen Sorry, but since I don't have access to a Windows system running mingw64, I can't say much about how to install packages on that system. Robert mentioned that you may possibly use conda install csvkit, but I obviously have no way of testing this myself. – Kusalananda Jun 21 '22 at 20:04
  • @JohanAndresen I don't have a Windows system either, however, are you sure you're activating your conda environment? If so, can you simply install csvkit with conda? If that works, you can forget about installing pip. – r_31415 Jun 21 '22 at 21:33
  • @RobertSmith do you suggest I run Anaconda prompt? (I would like to use bit bash). Or what do you mean by "activating your conda environment"? After I pip install csvkit, and csvkit --help, Anaconda prompt displays 'csvkit' is not recognized as an internal or external command, operable program or batch file. – Johan Jun 22 '22 at 08:59
  • Upon running $ csvkit --help mingw64 displays bash: csvkit: command not found. Have you any idea what to do? I added Anaconda\Scripts\ to PATH such that pip installed csvkit, which appears in the output of pip list "listing installed packages". – Johan Jun 22 '22 at 11:44
  • @JohanAndresen It was csvformat you wanted to use, wasn't it? There is no separate csvkit command. – Kusalananda Jun 22 '22 at 12:14
  • You are right, @Kusalanda. I also noticed this mistake, and it became clear that the installation worked. I wrote an answer to make this clear. Delightful! – Johan Jun 22 '22 at 12:19
0

These three steps installed csvkit, in reverse order:

  1. csvkit worked in git bash (mingw64) as indicated by

csvformat --help

  1. after installation using pip (pip was installed by Anaconda)

pip install csvkit

  1. made possible by adding the directory of pip.exe to PATH in the running session

PATH=$PATH:~/Anaconda3/Scripts

A more permanent solution requires adding the directory of pip.exe to e.g. .bashrc or .bash_profile, see e.g.:

How to correctly add a path to PATH

Extend $PATH variable in git bash under Windows

Johan
  • 399