Keeping an eye on my SharePoint crawls, I noticed some crawls were taking a long time.
The crawls were picking up a few items, but then I noticed the number of security updates.

Thanks to my friend Google, I found this posting on the subject.

A clip from the posting:

Question:

Why do security only crawls take so long?

Answer:

The time difference in crawl can be attributed to expansion of the SharePoint Group and also that the group is at the site collection level and affects items beyond the list. If a Sharepoint group has several thousand users at site collection level, you can see how this can be very expensive. Also, a large number of items within that site collection can add to the delay because new ACL changes will be pushed down to every item affected by the security change.

Question:

How can I work around this and prevent security only crawls from affecting incremental crawl times?

Answer:

Instead of users explicitly added to SharePoint groups, add AD groups instead. Managing adding/removing users from Active Directory security groups will not cause ACL changes within SharePoint. Because of this, no security only crawls will occur.

*Update*

Read this:

http://blogs.msdn.com/b/kaevans/archive/2013/05/06/clarifying-guidance-on-sharepoint-security-groups-versus-active-directory-domain-services-groups.aspx