This article is an attempt to provide the background information, tools and mechanisms to spot and correct Public Key Infrastructure-related issues for those who are setting-up Centrify Multi-factor Authentication or trying to enroll Identity Broker clients.
Public Key Infrastructure is at the heart of how many Internet and corporate infrastructure services are secured today and Centrify has always provided ways to make PKI simpler for organizations. A big example is adcert, this tool provides support for enterprise trust and auto-enrollment for Microsoft Certification Authority for UNIX, Linux and Mac.
After the introduction of Identity Service and Privilege Service and the advent of high-profile data breaches and industry guidance like PCI 3.2 Data Security Standard (Requirement 8, Sections 3, 10, 11 & 12), many organizations are rushing to implement Multi-factor Authentication. Another big milestone is the popularity of hybrid clouds; Centrify has introduced a new capability called Identity Broker, this new Linux Agent allows organizations to "enroll" in Centrify Privilege Service and to "bridge" multi-source enterprise directories like Active Directory, LDAP, Google Directory and Centrify directory.
All these scenarios make use of Public Key Infrastructure to establish the assurance that clients are talking to the right entity (non-repudiation) and encryption in transit is enforced. An important point to understand about every Centrify SaaS or on premises tenant is that it has an internal certification authority that is used for multiple uses including encryption, non-repudiation, mobile management and authentication.
PKI Trust and Multi-factor Authentication
With Centrify Identity Service and Privilege Service, it's possible for current users of Centrify Express or Centrify Standard Edition to implement MFA in a very quick and effective way; supporting both modern and legacy-based (RADIUS) solutions. In the platform's 16.10 release, Centrify proactively deprecated Integrated Windows Authentication (SPNEGO) over HTTP to exclusively use HTTPS.
The implication for users is that any interaction that used IWA (SPNEGO) required PKI trust in the authentication framework for MFA negotiation.
This means that framework after 16.10 looks like this:
As you can see from the framework above, there are 3 ways you can make sure the PKI requirements are satisfied:
- Enterprise Trust: This is the preferred method. Ideally, an organization has a properly-implemented PKI trust capability. Unfortunately, this is relatively obscure especially in the mid-market. A great benefit to organizations using Microsoft PKI is that Centrify DirectControl agents will take care of the enterprise trust automatically by bundling the Root and Intermediate CAs into the proper UNIX or Linux bundle.
- Public Trust: This was a bit expensive a while back, but the easiest way to make sure that PKI trust works out of the box is to use certificates issued by a public vendor like Verisign or GoDaddy.
- Tenant Trust: Each Centrify tenant will automatically create IWA certificates for all the Centrify connectors in the deployment. This means that customers can either manually, with DevOps or with Microsoft GPOs can set up a trust chain. This can be automated but requires a bit of work.
How to determine if your UNIX/Linux system is ready for MFA
Centrify provides a tool (adcdiag) that will allow users to spot issues with the MFA configuration. For example, in a Centrified system with an AD environment with Microsoft PKI, the root CA certificate is automatically downloaded to the /var/centrify/net/certs folder and appended to the bundle corresponding to the platform.
Here's a sample output from a Centrified system with Enterprise Trust:
This output is favorable because DirectControl (adclient) is making sure a lot of the moving parts are in place including making sure that any root or intermediate CA certificates are in the trust chain. The reason why this "just works" is because a few seconds after joining AD, and if the system is allowed to certificate auto-enrollment, the client will make sure all the proper certs are provisioned to the system and the CA bundle is updated. This makes this process work like plug and play.
In cases when there is no trust, then the ca-bundle has to be updated with the IWA trust certificate from the tenant. When you run the adcdiag, several checks will fail including this one:
If you inspect the file referenced by adcdiag, there will be the following information in this section:
"Error setting certificate verify locations" and this will point to the CA bundle for the platform (e.g. /etc/pki/tls/certs/ca-bundle.crt). There are several ways to solve this issue:
- Enterprise: Appending the root CA certificate in PEM format to the CA bundle file
- Public: Making sure the CA bundle is up-to date
- Tenant: Appending the IWA root ca in PEM format to the CA bundle file.
Fixing MFA CA Trust issues in UNIX/Linux platforms
You'll need to know:
- How to get the certificate in question
- The encoding of the certificate you're receiving
- The location of the bundle for the operating system in question
- For large production deployments, you'll need to use a viable distribution method
adcdiag failed in a CentOS 6 system. The issue is with the /etc/pki/tls/certs/ca-bundle.crt. I am working with a SaaS instance of Identity Service.
Locate the IWA Cert and Determine the Encoding
- In Admin Portal > Settings > Network > Centrify Connectors > click the connector > IWA Service and click "Download your IWA root CA certificate"
- Locate the file and try to open it with a text editor. If the text reads "--- begin certificate" you are dealing with a usable certificate.
- Save the file and transfer it to your target system (e.g. IWACert.crt)
Append the certificate to the CA bundle
- In the target system, concatenate the contents of the certificate file to the platform CA bundle. E.g.
$ sudo cat /home/user/IWACert.crt >> /etc/pki/tls/certs/ca-bundle.crtNote: there are OS utilities like "update-ca-trust" that perform this step the correct way. This is for illustration only.
- Re-run adcdiag and verify the results.
As you can see, the steps above won't scale in a large environment. This is why the preferred method is to have enterprise trust in place. Other ways to distribute certificate settings include scripts, DevOps solutions like Chef or Puppet and in Microsoft PKI scenarios, you can use Group Policy.
How to determine if your Windows system is ready for MFA
Windows systems may be easier to work with when it comes to Enterprise Trust but you have to be skillful to troubleshoot as well.
- Certificates MMC snap-in: This allows you to review all certificate store.
Note that you have to be a local administrator to view the computer certificate store and that Centrify will add certificates in the local store of systems running the Connector.
- Certutil: This is one of the most powerful tools around "certutil -viewstore root" will display the trusted root CAs.
Centrify Access Manager
This Microsoft management console provides the capability to perform an end-to-end testing in scenarios where DirectAuthorize Roles are being used for MFA. You'll need to be at least on version 2016.
This option is under right-click Direct Manage Access Manager > Test Centrify Cloud Connection.
Diagnostics and Centrify Logger Service
The DirectAuthorize applet provides a "troubleshooting" tab that enables advanced capabilities like Diagnostics or Log inspection.
The diagnostics functon has been enhanced to help identify or troubleshoot issues with Identity Platform, this functionality is available if you are running version 3.4 (suite 2017) and above,:
The Centrify logger service is installed with Centrify Server suite. You can add it to the Centrify Agent for Windows(tm) for advanced troubleshooting capabilities.
The Centrify Agent for Windows will provide you visual feedback when there's a PKI-related issue (see below) but internally it's checking the Certificates directory under \ProgramData\Centrify\DirectAuthorize for the binary blob that represents the tenant's certificate.
In this case, the same solutiona applies, but in this case, we're placing the certificate in the trust store for Windows.
Like we discussed before, in large Enterprises, ideally Enterprise or Public trust is set up with automation tools or Group Policy.
Microsoft provides a great article on the topic: https://technet.microsoft.com/en-us/library/cc772491(v=ws.11).aspx
Bottom-line: When attempting to configure MFA, don't forget this checklist:
- Is there a PKI trust between the system and the Centrify service?
- Can the system authenticate via Kerberos? (is it joined to the domain natively (Windows) or via Centrify (UNIX/Linux))
- Is the machine added to a Centrify role that allows for Computer Login?
- Are all the ports required for communication cleared (8443 or custom)?
PKI Trust and Identity Broker
Identity Broker is Centrify's newest capability that allows for multi-directory authentication in private or public clouds.
IB also requires trust for operations like enrollment.
Identifying issues with cdebug
Depending on the state of the Linux system (if the ca-bundle is non-existent, modified or outdated) the enrollment operation will fail. Let's look at a failed enrollment log using two PuTTY windows.
Window 1: /usr/share/centrifycc/bin/cdebug on Window 1: tail /var/centrify/centrifycc.log -f
Window 2: cenroll -t tenant.my.centrify.com -c [code] -F all -l Identity-Broker-Users
Window 2: Failed to initialize connection to Centrify identity platform: Failed to connect to Centrify identity platform
Dec 17 18:16:07 engcen6 cenroll: DEBUG <centrify/cloud.post> Failed
to post HTTP request: Post https://tenant.my.centrify.com/health/ping:
x509: certificate signed by unknown authority
This can be further verified with the cURL command:
$ curl https://tenant.my.centrify.com $ curl: (77) Problem with the SSL CA cert (path? access rights?)
In this particular case, my tenant is on-premises Privilege Service, so I can follow the instructions on this KB:
The steps are very similar to the ones outlined above. The strategy depends on the use case Enterprise, Public or Tenant trust is being used.
When trying to enroll, the output is very different:
Verbose: Platform detected: centos_6_6_standard Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Enrolling in Centrify identity platform https://bootcamp.my.centrify.com/ using registration code... Starting Centrify agent... Centrify agent started. Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Feature enabled: Application-to-Application Password Management Feature enabled: Centrify Agent Authentication Verbose: Restarting Centrify agent after enabled features... You have successfully enrolled in Centrify identity platform: https://tenant.my.centrify.com/ You may need to restart other services that rely upon PAM and NSS or simply reboot the computer for proper operations. Failure to do so may result in login problems for cloud users.
At Centrify capabilities change to provide ease of use and supportability. We hope this article can help you anticipate issues with your testing or setup. Ultimately, at the enterprise level, PKI is a vital capability that has to be taken seriously and designed to balance the people-process-technology triad.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.