QGIS uses OGR for accessing shapefiles. Looking at the OGR Shapefile driver page it says
Currently the OGR Shapefile driver only supports attribute indexes for looking up specific values in a unique key column. To create an attribute index for a column issue an SQL command of the form "CREATE INDEX ON tablename USING fieldname". To drop the attribute indexes issue a command of the form "DROP INDEX ON tablename". The attribute index will accelerate WHERE clause searches of the form "fieldname = value". The attribute index is actually stored as a mapinfo format index and is not compatible with any other shapefile applications.
So it depends on if your indexed attribute is unique.
As an alternative, you can also switch to a database like postgis, geopackage or spatialite which are designed for big data and indexing.