This article is an attempt to provide the background information, tools and mechanisms to spot and correct Public Key Infrastructure-related issues for those who are setting-up Centrify Multi-factor Authentication or trying to enroll Identity Broker clients.
Public Key Infrastructure is at the heart of how many Internet and corporate infrastructure services are secured today and Centrify has always provided ways to make PKI simpler for organizations. A big example is adcert, this tool provides support for enterprise trust and auto-enrollment for Microsoft Certification Authority for UNIX, Linux and Mac.
After the introduction of Identity Service and Privilege Service and the advent of high-profile data breaches and industry guidance like PCI 3.2 Data Security Standard (Requirement 8, Sections 3, 10, 11 & 12), many organizations are rushing to implement Multi-factor Authentication. Another big milestone is the popularity of hybrid clouds; Centrify has introduced a new capability called Identity Broker, this new Linux Agent allows organizations to "enroll" in Centrify Privilege Service and to "bridge" multi-source enterprise directories like Active Directory, LDAP, Google Directory and Centrify directory.
All these scenarios make use of Public Key Infrastructure to establish the assurance that clients are talking to the right entity (non-repudiation) and encryption in transit is enforced. An important point to understand about every Centrify SaaS or on premises tenant is that it has an internal certification authority that is used for multiple uses including encryption, non-repudiation, mobile management and authentication.
PKI Trust and Multi-factor Authentication
With Centrify Identity Service and Privilege Service, it's possible for current users of Centrify Express or Centrify Standard Edition to implement MFA in a very quick and effective way; supporting both modern and legacy-based (RADIUS) solutions. In the platform's 16.10 release, Centrify proactively deprecated Integrated Windows Authentication (SPNEGO) over HTTP to exclusively use HTTPS.
The implication for users is that any interaction that used IWA (SPNEGO) required PKI trust in the authentication framework for MFA negotiation.
This means that framework after 16.10 looks like this:
As you can see from the framework above, there are 3 ways you can make sure the PKI requirements are satisfied:
- Enterprise Trust: This is the preferred method. Ideally, an organization has a properly-implemented PKI trust capability. Unfortunately, this is relatively obscure especially in the mid-market. A great benefit to organizations using Microsoft PKI is that Centrify DirectControl agents will take care of the enterprise trust automatically by bundling the Root and Intermediate CAs into the proper UNIX or Linux bundle.
- Public Trust: This was a bit expensive a while back, but the easiest way to make sure that PKI trust works out of the box is to use certificates issued by a public vendor like Verisign or GoDaddy.
- Tenant Trust: Each Centrify tenant will automatically create IWA certificates for all the Centrify connectors in the deployment. This means that customers can either manually, with DevOps or with Microsoft GPOs can set up a trust chain. This can be automated but requires a bit of work.
How to determine if your UNIX/Linux system is ready for MFA
Centrify provides a tool (adcdiag) that will allow users to spot issues with the MFA configuration. For example, in a Centrified system with an AD environment with Microsoft PKI, the root CA certificate is automatically downloaded to the /var/centrify/net/certs folder and appended to the bundle corresponding to the platform.
Here's a sample output from a Centrified system with Enterprise Trust:
This output is favorable because DirectControl (adclient) is making sure a lot of the moving parts are in place including making sure that any root or intermediate CA certificates are in the trust chain. The reason why this "just works" is because a few seconds after joining AD, and if the system is allowed to certificate auto-enrollment, the client will make sure all the proper certs are provisioned to the system and the CA bundle is updated. This makes this process work like plug and play.
In cases when there is no trust, then the ca-bundle has to be updated with the IWA trust certificate from the tenant. When you run the adcdiag, several checks will fail including this one:
If you inspect the file referenced by adcdiag, there will be the following information in this section:
"Error setting certificate verify locations" and this will point to the CA bundle for the platform (e.g. /etc/pki/tls/certs/ca-bundle.crt). There are several ways to solve this issue:
- Enterprise: Appending the root CA certificate in PEM format to the CA bundle file
- Public: Making sure the CA bundle is up-to date
- Tenant: Appending the IWA root ca in PEM format to the CA bundle file.
Fixing MFA CA Trust issues in UNIX/Linux platforms
You'll need to know:
- How to get the certificate in question
- The encoding of the certificate you're receiving
- The location of the bundle for the operating system in question
- For large production deployments, you'll need to use a viable distribution method
adcdiag failed in a CentOS 6 system. The issue is with the /etc/pki/tls/certs/ca-bundle.crt. I am working with a SaaS instance of Identity Service.
Locate the IWA Cert and Determine the Encoding
- In Admin Portal > Settings > Network > Centrify Connectors > click the connector > IWA Service and click "Download your IWA root CA certificate"
- Locate the file and try to open it with a text editor. If the text reads "--- begin certificate" you are dealing with a usable certificate.
- Save the file and transfer it to your target system (e.g. IWACert.crt)
Append the certificate to the CA bundle
- In the target system, concatenate the contents of the certificate file to the platform CA bundle. E.g.
$ sudo cat /home/user/IWACert.crt >> /etc/pki/tls/certs/ca-bundle.crtNote: there are OS utilities like "update-ca-trust" that perform this step the correct way. This is for illustration only.
- Re-run adcdiag and verify the results.
As you can see, the steps above won't scale in a large environment. This is why the preferred method is to have enterprise trust in place. Other ways to distribute certificate settings include scripts, DevOps solutions like Chef or Puppet and in Microsoft PKI scenarios, you can use Group Policy.
How to determine if your Windows system is ready for MFA
Windows systems may be easier to work with when it comes to Enterprise Trust but you have to be skillful to troubleshoot as well.
- Certificates MMC snap-in: This allows you to review all certificate store.
Note that you have to be a local administrator to view the computer certificate store and that Centrify will add certificates in the local store of systems running the Connector.
- Certutil: This is one of the most powerful tools around "certutil -viewstore root" will display the trusted root CAs.
Centrify Access Manager
This Microsoft management console provides the capability to perform an end-to-end testing in scenarios where DirectAuthorize Roles are being used for MFA. You'll need to be at least on version 2016.
This option is under right-click Direct Manage Access Manager > Test Centrify Cloud Connection.
Diagnostics and Centrify Logger Service
The DirectAuthorize applet provides a "troubleshooting" tab that enables advanced capabilities like Diagnostics or Log inspection.
The diagnostics functon has been enhanced to help identify or troubleshoot issues with Identity Platform, this functionality is available if you are running version 3.4 (suite 2017) and above,:
The Centrify logger service is installed with Centrify Server suite. You can add it to the Centrify Agent for Windows(tm) for advanced troubleshooting capabilities.
The Centrify Agent for Windows will provide you visual feedback when there's a PKI-related issue (see below) but internally it's checking the Certificates directory under \ProgramData\Centrify\DirectAuthorize for the binary blob that represents the tenant's certificate.
In this case, the same solutiona applies, but in this case, we're placing the certificate in the trust store for Windows.
Like we discussed before, in large Enterprises, ideally Enterprise or Public trust is set up with automation tools or Group Policy.
Microsoft provides a great article on the topic: https://technet.microsoft.com/en-us/library/cc7724
Bottom-line: When attempting to configure MFA, don't forget this checklist:
- Is there a PKI trust between the system and the Centrify service?
- Can the system authenticate via Kerberos? (is it joined to the domain natively (Windows) or via Centrify (UNIX/Linux))
- Is the machine added to a Centrify role that allows for Computer Login?
- Are all the ports required for communication cleared (8443 or custom)?
PKI Trust and Identity Broker
Identity Broker is Centrify's newest capability that allows for multi-directory authentication in private or public clouds.
IB also requires trust for operations like enrollment.
Identifying issues with cdebug
Depending on the state of the Linux system (if the ca-bundle is non-existent, modified or outdated) the enrollment operation will fail. Let's look at a failed enrollment log using two PuTTY windows.
Window 1: /usr/share/centrifycc/bin/cdebug on Window 1: tail /var/centrify/centrifycc.log -f
Window 2: cenroll -t tenant.my.centrify.com -c [code] -F all -l Identity-Broker-Users
Window 2: Failed to initialize connection to Centrify identity platform: Failed to connect to Centrify identity platform
Dec 17 18:16:07 engcen6 cenroll: DEBUG <centrify/cloud.post> Failed
to post HTTP request: Post https://tenant.my.centrify.com/health/ping:
x509: certificate signed by unknown authority
This can be further verified with the cURL command:
$ curl https://tenant.my.centrify.com $ curl: (77) Problem with the SSL CA cert (path? access rights?)
In this particular case, my tenant is on-premises Privilege Service, so I can follow the instructions on this KB:
The steps are very similar to the ones outlined above. The strategy depends on the use case Enterprise, Public or Tenant trust is being used.
When trying to enroll, the output is very different:
Verbose: Platform detected: centos_6_6_standard Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Enrolling in Centrify identity platform https://bootcamp.my.centrify.com/ using registration code... Starting Centrify agent... Centrify agent started. Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Feature enabled: Application-to-Application Password Management Feature enabled: Centrify Agent Authentication Verbose: Restarting Centrify agent after enabled features... You have successfully enrolled in Centrify identity platform: https://tenant.my.centrify.com/ You may need to restart other services that rely upon PAM and NSS or simply reboot the computer for proper operations. Failure to do so may result in login problems for cloud users.
At Centrify capabilities change to provide ease of use and supportability. We hope this article can help you anticipate issues with your testing or setup. Ultimately, at the enterprise level, PKI is a vital capability that has to be taken seriously and designed to balance the people-process-technology triad.
Amazon AWS is at the heart of many of our customers workloads. Last year I started a series of tech blogs to discuss how to leverage Centrify's product portfolio to secure your AWS assets.
This year, I've had the opportunity to review the AWS Security Best Practices document and in this new series we'll provide guidance on how to implement controls to meet or exceed the Shared Responsibility Model.
About the Shared Responsibility Model
The concept is very straightforward. Amazon AWS will implement controls to provide assurance for confidentiality (e.g. encryption at rest and in transit), integrity (transaction trust), availability (redundancy of hardware, power, etc), however, depending on your business requirements, you may need to add additional controls to increase your security posture or to provide assurances to your customers beyond what's offered by AWS.
Amazon AWS Defines a "Shared Responsibility" model that has the following scope
- Infrastructure Services: Controls that apply to IaaS services like EC2, VPCs and Block Storage.
- Container Services: Controls that apply to PaaS servides like RDS Database, EMR MapReduce or Elastic Beanstalk
- Abstracted Services: Controls that apply to Services like S3 Storage, SES SMTP, etc.
In this article we'll focus on how to use Centrify Privilege Service to secure access to Windows AMIs.
The Challenge: Extend Existing Enterprise Identity and Access Management in Public Clouds
In a previous entry of this series, we discussed the additional controls that can be implemented on top of Amazon's IAM and cryptography-based controls. However, organizations may already have capabilities for Centralized Identity which is at the heart of strong access controls. This may mean that organizations need to find ways to extend this infrastructure out to public clouds like Amazon. This may imply:
- Re-creation of infrastructure (e.g. standing-up Active Directory or LDAP-like infrastructure)
- Network extension (e.g. treating the public cloud like a branch by implementing site-to-site VPNs)
- Capability duplication (e.g. implementing software and services in AWS)
In this article we'll discuss how organizations can leverage Centrify Privilege Service and the new Linux Agent (Identity Broker) to secure Linux instances and extend Enterprise Identity out to public clouds without the need of the additional overhead.
Centrify Privilege Service
Privilege Service is Centrify's answer to the traditional "password-driven" use cases (the industry refers as Shared Account Password Management, Privilege Session Management, etc); however unlike other solutions, there are several capabilities areas that set it apart from the traditional "Password Vault"
- Flexible deployment: Both as SaaS and On Premises (in fact, it was designed as a SaaS solution first)
- Identity Sourcing and Federation built-in (includes Identity Service, this means support for AD, LDAP, Google and others plus simplified SAML-based federation and 3K+ ready web and mobile apps)
- Policy, Workflow and MFA Engine: Policies for time and geo-location, RBAC, step-up authentication and Multi-factor including Smartcard, plus a built-in access request system (+ServiceNow integration)
- Infrastructure Extensibility: Privilege Service can be extended to any network using a Windows-based Centrify Connector via web protocols.
New Linux Agent
The new Linux agent takes advantage of a capability called "Identity Broker" this allows the bridging of identities known to Centrify Privilege Service; this is done via the Centrify connector. This means that the overhead of duplicating enterprise identity infrastructure or extending the enterprise to the public network can be avoided in this particular use case; all that is required is to deploy a Centrify connector wherever you want to extend Password-related services and Linux authentication. Let's show an illustration.
Company X needs to provide identity-based reporting and attestation of who has access to their public cloud EC2 instances in AWS; their primary identity source is AD. They could have used any model (independent forest, one-way trust + site2site VPN or RODC) to extend AD to AWS or they could have deployed Amazon IAM roles and used SSH keys; but any of these models implied additional overhead. With CPS and Identity Broker, all they did is this:
With this model, CompanyX not only can accomodate their corporate IT users, but external entities as well. And as we discussed before, password, session and additional services are consolidated as well, both on premises and on any public cloud. Plus
- No need to deploy "jumpboxes" to the private clouds (limits exposure)
- Shared account passwords for local accounts (Linux, Windows) or databases (like Amazon RDS) can be controlled
- Access Request provides the assurance of "documented approvals" to sensitive systems
- Deploy MFA or access only from the OnPrem network as additional controls.
The client architecture is using UNIX frameworks like Name Service Switch (Identity) and Pluggable Authentication Modules (Auth). It also supports offline login as well as direct or proxy-based user/password or OAUTH-based enrollment codes (very useful for automation). This client implements the CLI tookit for setting, retrieving or deleting credentials under management.
Following on the legacy of Server Suite, the new agent generates identity like DirectControl in workstation mode.
Login - user's short name (must use the fully-qualified name the first time)
UID - auto-generated based on the GUID
Primary group - auto-private (same as UID)
GECOS - the display name in Centrify Identity Service
Home/Shell - configurable in the Settings tab.
Since most public clouds don't need legacy identity (for services like NFS), this makes the client very lightweight.
There's an implied expectation of DevOps "friendliness" when a private or public cloud solution is implemented. The new Linux agent leverages enrollment codes for this capability.
Centrify provides a sample AWS script that can be used with enrollment codes in the User Data field or with any other framework like OpsWorks (see attachment)
Privilege Service can accomodate several identities, including:
Note that it can accommodate identities from Active Directories without any trust relationships.
These identities can be aggregated using Identity Service Roles:
Roles, in turn can be assigned the AgentAuth privilege on a Linux Resource:
As you can see the model works like this:
Users or groups from Identity Sources are added to CIS Roles that are granted the AgentAuth right in CPS.
- cinfo – obtain information about the agent
- cenroll – enroll the identity platform and enable features
- cunenroll – leave the platform and optionally delete resource and any managed accounts
- cflush – flush the local cache
- csetaccount – add a managed account for the resource
- cgetaccount – obtain a managed account’s password
- cdelaccount - delete a managed account's password
To test-drive the new Linux agent and to see how it can secure your public-cloud Linux instances, request a CPS trial today: https://www.centrify.com/free-trial/privilege-serv