Is there a limitation on the amount of data Directory Lister can scan?
There is no limitation on the amount of data that can be scanned. However, for HTML, text, CSV and Excel listings, Directory Lister creates a temporary SQLite database which is initially stored in memory, but at certain size it's flushed to disk. So this can hit performance a little bit.
Alternatively for such purposes there is a database output which currently supports Microsoft SQL Server, MySQL, PostgreSQL, Firebird, SQLite and Microsoft Access databases. For database output, only one directory is stored in memory.