etldx

Splits a domain name into its subdomain, registrable label and public suffix, using the Public Suffix List. You supply the list, it is neither downloaded nor bundled.

val psl = PublicSuffixList(resources.openRawResource(R.raw.public_suffix_list))

val domain = psl.split("forums.bbc.co.uk")
domain.subdomain          // "forums"
domain.label              // "bbc"
domain.publicSuffix       // "co.uk"
domain.registrableDomain  // "bbc.co.uk"

psl.split("www.worldbank.org.kg").registrableDomain     // "worldbank.org.kg"
psl.split("waiterrant.blogspot.com").registrableDomain  // "waiterrant.blogspot.com"
psl.isPublicSuffix("co.uk")                             // true

Input

The only supported input is a domain name. URLs and IP addresses are rejected, so pull the host out first with java.net.URI, okhttp3.HttpUrl or android.net.Uri. split and isPublicSuffix throw InvalidDomainNameException for anything that is not a name.

Names are lower cased and matched in Punycode, but results are sliced out of the input, so example.香港 stays Unicode and example.xn--j6w193g stays Punycode.

Supplying the list

The constructor takes whatever shape the text arrives in. Readers and streams are read fully and closed.

PublicSuffixList(resources.openRawResource(R.raw.public_suffix_list))  // Android
PublicSuffixList(response.body!!.charStream())                         // OkHttp
PublicSuffixList(response.bodyAsText())                                // Ktor
PublicSuffixList(GZIPInputStream(file.inputStream()))                  // local copy
PublicSuffixList("com\nco.uk\n*.ck\n!www.ck")                          // tests
PublicSuffixList(lines)                                                // Sequence<String>

The list lives at https://publicsuffix.org/list/public_suffix_list.dat. Downloading, caching and refreshing it is up to you. Instances are immutable and thread safe, so build one and keep it.

Options

PublicSuffixList(source, Options.ICANN_ONLY)  // foo.blogspot.com -> blogspot.com
PublicSuffixList(source, Options.STRICT)      // unknown TLDs are rejected
PublicSuffixList(source, Options(sections = setOf(Section.PRIVATE)))

sections picks which half of the list to load. Private rules are submitted by domain holders themselves, such as blogspot.com and github.io; excluding them treats those as ordinary domains. Rules outside any section marker are always loaded, so a hand written list needs no markers.

acceptUnknownSuffixes decides what happens when the TLD is not in the list. On by default, so shop.example.newtld gives example.newtld with isKnownSuffix = false. Turn it off when you want lookups to double as validation: split then throws on an unknown TLD and isPublicSuffix is false.

Notes

• Malformed rules are skipped instead of throwing, since a list downloaded at runtime should not be able to crash the app.
• *.foo.com does not imply that foo.com is a public suffix, as the format spec requires.

Build

./gradlew assemble

Test

./gradlew test

License

MIT