Out of the box, anonymous access is a pretty cool idea but breaks several things. Just to help those who may think about anonymous access (defined as users that don't login at all on something), here is a list of things that are altered:
1. Workflows - worfklows that are set to automatically run when you create a new item will break if you turn on anonymous access. Not only will the workflow not run, but here's what will typically happen: a user will click "new", fill out your sharepoint form, and then they click OK. When they do, they get an error message saying that there is a Server Out of Memory and think that something went wrong, so they try to fill it out again. What the anonymous user DOESN'T know is that their information WAS submitted but the workflow didn't run. The error message that they saw was in regard to the workflow. If this happens, the primary way people are fixing it is to create a new list as a copy of the current list, copy all the data over from the old list to the new one, and delete the old list. You CANNOT just delete the workflow...certain pieces of the workflow are not deleted when you delete a workflow and THOSE are the things that are breaking the workflow. Some people have figured out another solution but this seems to be the main way to fix it.
2. Surveys - SharePoint surveys are already a very limited template but they get further limited with anonymous access in that you cannot use branching logic, page breaks, or limiting people to only filling out a survey 1 time. This doesn't mean it's not useful, it's just very limited in its applications.
3. Libraries - Anonymous users are not meant to access libraries, so just get that out of your head now. Technically, you can use a URL hack to allow them access but they won't be able to add and upload documents or anything.
4. InfoPath Forms - anonymous users can't access infopath forms unless you have forms-based authentication (which supplies a default username/password if none exists when you turn on anonymous access).
5. Edit or Read Your Own - these settings typically only allow someone to see or edit their own stuff...well, this is impossible with anonymous users because there's no way to tell one anonymous user from another and, thus, it will not show anything for anonymous users. This can sometimes be a benefit and other times a problem.
That's all for now; if you find anything else, please let me know and I'll throw it up here.