As mentioned you and other answers have already surmised, aside from acquiring customers, learning their needs, etc, etc, acquiring the necessary data to deliver useful analysis to your customers may be one of your most challenging efforts. From a business perspective, though, the most valuable thing that you have to offer is your service and knowledge of using software and data, not which software you choose to use.
The most valuable (and costly) thing that ArcGIS Business Analyst provides is the bundled data (like demographics, consumer spending, business locations), as well as the pre-built functions to use the data. Many of the functions can be recreated through other software and database applications, though. If you can afford to purchase additional data, you may be able to plug this data into other systems (with some effort on your part) and be on your way.
For open-source software, consider QGIS which is a great map viewer with some good analysis functions. For your database, consider PostGIS, the spatial extension to PostgreSQL, which has hundreds of GIS functions built into the database that you can use for analysis, and then view the results in QGIS. Yes, it will take you some time to learn these new technologies, but you can learn the basics at the BostonGIS website, along with questions tagged with business, business-intelligence, and geomarketing) here on GIS.Stackexchange, and others. With this combination of tools, I think you will find many of the functions needed to perform the types of analysis you are looking for. Given that the software is open source, you then also have the ability to modify, enhance, or hire consultants to enhance the software, and eventually even contribute your knowledge back to the greater open-source community.
As both @Fetzer and @MappaGnosis have mentioned, you can dramatically lower your cost of entry for software by going the open-source route for not only GIS, but also many of your other daily operations (like office, presentations, email, etc). Good Information/data will always remain a key need, though.
I recently watched a couple of video's by @PaulRamsey that I think you would benefit by watching, regarding the open-source business model: