1

i need to send Http request to a Hadoop Cluster with Kerberos auth and i want to authenticate with a Keytab. I found a library HttpClient by Apache and i wrote this code to get the HttpClient:

    private static CloseableHttpClient getHttpClient() {
    CredentialsProvider provider = new BasicCredentialsProvider();
    provider.setCredentials(
            AuthScope.ANY,
            new UsernamePasswordCredentials("YarnManager", "YarnManager")
    );
    return HttpClientBuilder.create()
            .setDefaultCredentialsProvider(provider)
            .build();
}

But i can only authenticate after doing Kinit command on my KeyTab. How can i pass the path to my keytab in httpclient? Do you know other libraries to do that?

Lorenzo Di Giacomo
  • 129
  • 3
  • 13

2 Answers2

1

About Apache HttpClient: quoting Hadoop and Kerberos, the Madness beyond the Gate by Steve Loughran (from when he had to maintain hadoop-auth library)

The Apache HTTP Client/http components have a well-deserved reputation for being great libraries to work with remote HTTP servers.
Should you use them for Kerberos/SPNEGO authenticated applications?

No.
(...)
Don't waste time or make things worse: go with the JDK libraries

About Kerberos auth over HTTP: the standard is to use SPNego which is implemented by every decent HTTP client in any language. And by any serious HTTP server or proxy, although the setup is more complicated server-side.

The problem about Java clients is that, quoting Steve Lougran again,

the public Java APIs are brittle across versions and JDKs ... Releases within a Java version may break the internals and/or the public API's behaviour.

That may explain why the "native" Java HTTP client is more robust that the Apache one, when it comes to the Java implementation of Kerberos i.e. JAAS -- that's what you need to configure if you want to be able to use a keytab.

Now, the Java implementation of Kerberos is not only brittle, it's incomplete (no way to write into the ticket cache, no way to create renewable tickets, etc) and not well documented -- quoting S.L. once more

JAAS is a nightmare from the Enterprise Java Bean era, one which surfaces from the depths to pull the unwary under
(...)
a single jaas.conf file can have multiple contexts ... Different contexts can have different login/auth mechanisms, including Kerberos and LDAP

Fun fact: actually a single context may define multiple mechanisms (1st entry is tried, if unsuccessful 2nd is tried etc) -- and for each mechanism, multiple ways to get credentials. See these posts for the syntax & for more reference:
https://stackoverflow.com/a/60507547/5162372
https://stackoverflow.com/a/45776704/5162372

AFAIK the default JAAS behaviour for SPNego is "look into the system cache (as defined by $KRB5CCNAME or by /etc/krb5.conf or by the hard-coded default); otherwise prompt for user/password unless the prompting is blocked by the JVM security policy (which is the default since the JVM cannot guess whether it needs an UI prompt or a console prompt)".

You will need to override that to create a private, in-memory ticket from a keytab; and also need some tweaks that are not really well documented -- cf. that post for example
https://stackoverflow.com/a/42506620/5162372

The post above focuses on JDBC but also applies to SPNego with two caveats:

  • the JAAS section must be labeled com.sun.security.jgss.krb5.initiate
  • there's an extra debug flag -Dsun.security.spnego.debug=true

Also, that "tutorial" from Oracle does not cover the keytab option but may be useful for context:
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/lab/part6.html

Samson Scharfrichter
  • 8,884
  • 1
  • 17
  • 36
0

I implemented a similar thing recently, but instead of Java, it was in C++.

There was no straightforward way to get around that. Doing a kinit was left as a responsibility of the user.

displayName
  • 13,888
  • 8
  • 60
  • 75