I think the situation is familiar to everyone. This happens due to ill-management or bad estimates, and this is often a problem.
The company I work in has a certain policy about this. If someone has no work, this is the problem of the whole company, and they should inform the manager, the team leader and our boss about this beforehand (at least 1-2 days). So when the situation is encountered, it can be solved.
Even if there is no such policy in your company, this is the best thing to do. You can't be sure whether others know about your problem until you talk to them. Communicate. Ask if you don't know (or if you aren't sure). It also shows that you are a responsible person, and you won't conceal any problems.
The second step (in my company) is to see if anyone else needs help. If you have spare time, and someone else can meet the schedule, you could do some of their work. And again, the management must know what you are doing.
If there is no chance to get work for now, the best thing you can do is self-improvement. Read books, articles, StackExchange sites (highly recommended), etc.
Another solution is a vacation (which, in most cases, isn't likely to happen). My boss had asked (yes, asked) me to take a paid vacation (this doesn't reduce the quantity of days that I can still take), until there is some work for me. And when there was work, he called me beforehand and asked to come back.