2025 State of the Internet Report: Open Directories Time to Live

This Censys research note examines the lifespans of open web directories observed on HTTP in April, using TLSH hashing of HTTP bodies to measure content similarity. The study finds that while 50% of open directories are visible on the network for one day or less, their content tends to persist longer (median about three days), highlighting the importance of combining real-time network visibility with content-based tracking for effective investigations; the report also provides a Censys query to locate open directories.