This article is an attempt to provide the background information, tools and mechanisms to spot and correct Public Key Infrastructure-related issues for those who are setting-up Centrify Multi-factor Authentication or trying to enroll Identity Broker clients.
Public Key Infrastructure is at the heart of how many Internet and corporate infrastructure services are secured today and Centrify has always provided ways to make PKI simpler for organizations. A big example is adcert, this tool provides support for enterprise trust and auto-enrollment for Microsoft Certification Authority for UNIX, Linux and Mac.
After the introduction of Identity Service and Privilege Service and the advent of high-profile data breaches and industry guidance like PCI 3.2 Data Security Standard (Requirement 8, Sections 3, 10, 11 & 12), many organizations are rushing to implement Multi-factor Authentication. Another big milestone is the popularity of hybrid clouds; Centrify has introduced a new capability called Identity Broker, this new Linux Agent allows organizations to "enroll" in Centrify Privilege Service and to "bridge" multi-source enterprise directories like Active Directory, LDAP, Google Directory and Centrify directory.
All these scenarios make use of Public Key Infrastructure to establish the assurance that clients are talking to the right entity (non-repudiation) and encryption in transit is enforced. An important point to understand about every Centrify SaaS or on premises tenant is that it has an internal certification authority that is used for multiple uses including encryption, non-repudiation, mobile management and authentication.
PKI Trust and Multi-factor Authentication
With Centrify Identity Service and Privilege Service, it's possible for current users of Centrify Express or Centrify Standard Edition to implement MFA in a very quick and effective way; supporting both modern and legacy-based (RADIUS) solutions. In the platform's 16.10 release, Centrify proactively deprecated Integrated Windows Authentication (SPNEGO) over HTTP to exclusively use HTTPS.
The implication for users is that any interaction that used IWA (SPNEGO) required PKI trust in the authentication framework for MFA negotiation.
This means that framework after 16.10 looks like this:
As you can see from the framework above, there are 3 ways you can make sure the PKI requirements are satisfied:
- Enterprise Trust: This is the preferred method. Ideally, an organization has a properly-implemented PKI trust capability. Unfortunately, this is relatively obscure especially in the mid-market. A great benefit to organizations using Microsoft PKI is that Centrify DirectControl agents will take care of the enterprise trust automatically by bundling the Root and Intermediate CAs into the proper UNIX or Linux bundle.
- Public Trust: This was a bit expensive a while back, but the easiest way to make sure that PKI trust works out of the box is to use certificates issued by a public vendor like Verisign or GoDaddy.
- Tenant Trust: Each Centrify tenant will automatically create IWA certificates for all the Centrify connectors in the deployment. This means that customers can either manually, with DevOps or with Microsoft GPOs can set up a trust chain. This can be automated but requires a bit of work.
How to determine if your UNIX/Linux system is ready for MFA
Centrify provides a tool (adcdiag) that will allow users to spot issues with the MFA configuration. For example, in a Centrified system with an AD environment with Microsoft PKI, the root CA certificate is automatically downloaded to the /var/centrify/net/certs folder and appended to the bundle corresponding to the platform.
Here's a sample output from a Centrified system with Enterprise Trust:
This output is favorable because DirectControl (adclient) is making sure a lot of the moving parts are in place including making sure that any root or intermediate CA certificates are in the trust chain. The reason why this "just works" is because a few seconds after joining AD, and if the system is allowed to certificate auto-enrollment, the client will make sure all the proper certs are provisioned to the system and the CA bundle is updated. This makes this process work like plug and play.
In cases when there is no trust, then the ca-bundle has to be updated with the IWA trust certificate from the tenant. When you run the adcdiag, several checks will fail including this one:
If you inspect the file referenced by adcdiag, there will be the following information in this section:
"Error setting certificate verify locations" and this will point to the CA bundle for the platform (e.g. /etc/pki/tls/certs/ca-bundle.crt). There are several ways to solve this issue:
- Enterprise: Appending the root CA certificate in PEM format to the CA bundle file
- Public: Making sure the CA bundle is up-to date
- Tenant: Appending the IWA root ca in PEM format to the CA bundle file.
Fixing MFA CA Trust issues in UNIX/Linux platforms
You'll need to know:
- How to get the certificate in question
- The encoding of the certificate you're receiving
- The location of the bundle for the operating system in question
- For large production deployments, you'll need to use a viable distribution method
adcdiag failed in a CentOS 6 system. The issue is with the /etc/pki/tls/certs/ca-bundle.crt. I am working with a SaaS instance of Identity Service.
Locate the IWA Cert and Determine the Encoding
- In Admin Portal > Settings > Network > Centrify Connectors > click the connector > IWA Service and click "Download your IWA root CA certificate"
- Locate the file and try to open it with a text editor. If the text reads "--- begin certificate" you are dealing with a usable certificate.
- Save the file and transfer it to your target system (e.g. IWACert.crt)
Append the certificate to the CA bundle
- In the target system, concatenate the contents of the certificate file to the platform CA bundle. E.g.
$ sudo cat /home/user/IWACert.crt >> /etc/pki/tls/certs/ca-bundle.crtNote: there are OS utilities like "update-ca-trust" that perform this step the correct way. This is for illustration only.
- Re-run adcdiag and verify the results.
As you can see, the steps above won't scale in a large environment. This is why the preferred method is to have enterprise trust in place. Other ways to distribute certificate settings include scripts, DevOps solutions like Chef or Puppet and in Microsoft PKI scenarios, you can use Group Policy.
How to determine if your Windows system is ready for MFA
Windows systems may be easier to work with when it comes to Enterprise Trust but you have to be skillful to troubleshoot as well.
- Certificates MMC snap-in: This allows you to review all certificate store.
Note that you have to be a local administrator to view the computer certificate store and that Centrify will add certificates in the local store of systems running the Connector.
- Certutil: This is one of the most powerful tools around "certutil -viewstore root" will display the trusted root CAs.
Centrify Access Manager
This Microsoft management console provides the capability to perform an end-to-end testing in scenarios where DirectAuthorize Roles are being used for MFA. You'll need to be at least on version 2016.
This option is under right-click Direct Manage Access Manager > Test Centrify Cloud Connection.
Diagnostics and Centrify Logger Service
The DirectAuthorize applet provides a "troubleshooting" tab that enables advanced capabilities like Diagnostics or Log inspection.
The diagnostics functon has been enhanced to help identify or troubleshoot issues with Identity Platform, this functionality is available if you are running version 3.4 (suite 2017) and above,:
The Centrify logger service is installed with Centrify Server suite. You can add it to the Centrify Agent for Windows(tm) for advanced troubleshooting capabilities.
The Centrify Agent for Windows will provide you visual feedback when there's a PKI-related issue (see below) but internally it's checking the Certificates directory under \ProgramData\Centrify\DirectAuthorize for the binary blob that represents the tenant's certificate.
In this case, the same solutiona applies, but in this case, we're placing the certificate in the trust store for Windows.
Like we discussed before, in large Enterprises, ideally Enterprise or Public trust is set up with automation tools or Group Policy.
Microsoft provides a great article on the topic: https://technet.microsoft.com/en-us/library/cc7724
Bottom-line: When attempting to configure MFA, don't forget this checklist:
- Is there a PKI trust between the system and the Centrify service?
- Can the system authenticate via Kerberos? (is it joined to the domain natively (Windows) or via Centrify (UNIX/Linux))
- Is the machine added to a Centrify role that allows for Computer Login?
- Are all the ports required for communication cleared (8443 or custom)?
PKI Trust and Identity Broker
Identity Broker is Centrify's newest capability that allows for multi-directory authentication in private or public clouds.
IB also requires trust for operations like enrollment.
Identifying issues with cdebug
Depending on the state of the Linux system (if the ca-bundle is non-existent, modified or outdated) the enrollment operation will fail. Let's look at a failed enrollment log using two PuTTY windows.
Window 1: /usr/share/centrifycc/bin/cdebug on Window 1: tail /var/centrify/centrifycc.log -f
Window 2: cenroll -t tenant.my.centrify.com -c [code] -F all -l Identity-Broker-Users
Window 2: Failed to initialize connection to Centrify identity platform: Failed to connect to Centrify identity platform
Dec 17 18:16:07 engcen6 cenroll: DEBUG <centrify/cloud.post> Failed
to post HTTP request: Post https://tenant.my.centrify.com/health/ping:
x509: certificate signed by unknown authority
This can be further verified with the cURL command:
$ curl https://tenant.my.centrify.com $ curl: (77) Problem with the SSL CA cert (path? access rights?)
In this particular case, my tenant is on-premises Privilege Service, so I can follow the instructions on this KB:
The steps are very similar to the ones outlined above. The strategy depends on the use case Enterprise, Public or Tenant trust is being used.
When trying to enroll, the output is very different:
Verbose: Platform detected: centos_6_6_standard Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Enrolling in Centrify identity platform https://bootcamp.my.centrify.com/ using registration code... Starting Centrify agent... Centrify agent started. Verbose: Trying to connect to Centrify identity platform [https://tenant.my.centrify.com/] without a proxy... Feature enabled: Application-to-Application Password Management Feature enabled: Centrify Agent Authentication Verbose: Restarting Centrify agent after enabled features... You have successfully enrolled in Centrify identity platform: https://tenant.my.centrify.com/ You may need to restart other services that rely upon PAM and NSS or simply reboot the computer for proper operations. Failure to do so may result in login problems for cloud users.
At Centrify capabilities change to provide ease of use and supportability. We hope this article can help you anticipate issues with your testing or setup. Ultimately, at the enterprise level, PKI is a vital capability that has to be taken seriously and designed to balance the people-process-technology triad.
Amazon AWS is at the heart of many of our customers workloads. Last year I started a series of tech blogs to discuss how to leverage Centrify's product portfolio to secure your AWS assets.
This year, I've had the opportunity to review the AWS Security Best Practices document and in this new series we'll provide guidance on how to implement controls to meet or exceed the Shared Responsibility Model.
About the Shared Responsibility Model
The concept is very straightforward. Amazon AWS will implement controls to provide assurance for confidentiality (e.g. encryption at rest and in transit), integrity (transaction trust), availability (redundancy of hardware, power, etc), however, depending on your business requirements, you may need to add additional controls to increase your security posture or to provide assurances to your customers beyond what's offered by AWS.
Amazon AWS Defines a "Shared Responsibility" model that has the following scope
- Infrastructure Services: Controls that apply to IaaS services like EC2, VPCs and Block Storage.
- Container Services: Controls that apply to PaaS servides like RDS Database, EMR MapReduce or Elastic Beanstalk
- Abstracted Services: Controls that apply to Services like S3 Storage, SES SMTP, etc.
In this article we'll focus on how to use Centrify Privilege Service to secure access to Windows AMIs.
How is access to Amazon Windows AMIs secured today?
The AWS Security Best Practices document specifies the following information:
At a basic level, this just means that beyond providing the ability to decrypt the administrator password based on your private key, it's up to the customer to deploy additional controls (including x.509 authentication, Active Directory or local accounts).
For large organizations, both x.509 or local accounts may create an additional identity silo; this means that Active Directory (either an extension of the on-prem directory or an instance running in AWS is the main option).
Centrify Privilege Service and Centrify Server Suite provide the flexibility to implement these controls and we'll cover more in subsequent posts, now let's explore the benefits of combining Privilege Service with the Centrify Agent for Windows for Centralized Sessions and Windows MFA
Privilege Session Brokering facilitated by CPS provides these benefits:
- Centralized Identity Source: You can leverage your existing on-premises Active Directory to authenticate users accessing EC2 Windows AMIs.
- Centralized Session Initiation: This control centralizes all RDP sessions from a central management perspective. With CPS, this capability is enhanced by providing watch and terminate options (session proctoring) and session recording (capture and replay).
- Limited Exposure: by not using public IP addresses for your Windows systems, you are limiting exposure to attacks.
- Additional controls: CPS's ability to implement RBAC, Geo/time fencing can meet or exceed policy requirements.
- Password Services: Shared account password management for local Windows and UNIX accounts; AD accounts, Oracle and SQL databases, Windows services and scheduled tasks.
Multi-Factor Authentication with the Centrify Agent for Windows provides these benefits:
- You can enforce MFA using different factors like: Mobile Authenticator, OATH OTP, Legacy (e.g. SecurID), Phone factor, SMS, e-mail or security question.
- Contexts: You can limit this to login or privilege elevation (if you're using Centrify Server Suite zones technology - will be covered later).
- Enhanced controls: If you're allowing direct connectivity, users must provide MFA and authenticate with their AD credentials.
- Offline Access and Rescue Rights: Leverate OATH OTP codes for MFA in case there's no access to Centrify services
With Centrify Privilege Service + the Windows MFA capabilities enabled by the Centrify Agent for Windows we can secure Windows AMIs by providing ways to leverage your existing AD infrastructure to centralize sessions, provide SAPM services and provide MFA enforced at the local level.
In this demo we show how we can centralize session origination using Centrify Privilege Service, how we can use a shared account or log in manually. We'll configure Windows MFA for a demo user and we'll demonstrate how the control is enforced via the jumpbox or if the user chooses to connect to the Windows AMI directly.
This is a continuation of our previous article, in which we discussed how to eliminate the poor practice of sharing the root, administrator (or any other privileged account) across UNIX, Linux and Windows platforms using Centrify Standard Edition.
We build on this knowledge to tackle a bigger challenge: privileged execution of individual apps tied to session capture and replay.
Why implement granular Role-Based Access?
Prospects and customers come to us because of one or more of these issues:
In UNIX & Linux systems:
- They use sudo, but realize that there are challenges related to the administration and reporting of privileges based on that model.
- Privileged users end up doing this "sudo su -" or "dzdo su -" this makes it hard to truly detect who performed what actions in critical systems.
- There are poorly written but critical apps that require Administrator privileges to run, this means that a large population of users have admin rights in their PCs.
- There are too many members in privileged groups in AD just to be able to perform simple tasks.
- You are using Windows 2012 or Windows 8 and the "quick and easy" privileged elevation provided by Centrify's DirectAuthorize for Windows (New Desktop) is not available in these platforms.
Across all platforms
Organizations may have adopted a control (such as a password vault), and although now they have a better handle of who has access to a privileged account (and can approve/deny access) and the passwords are rotated, they realized that:
- They can't pass tougher audits that require the implementation of a strict Role Based Access Control
- Certain actions can't still be accounted for (some folks bypass the vault system and connect directly)
- Costs for added capabilities and users are growing exponentially
- The approvals process is too lax due to the fact that a lot of users need privileged actions as part of their job (even for simple operational tasks)
- Due to costs, the vault system is not replicated in Dev, QA environments and production processes are not uniform across the board.
Although there's been a significant investment in the capability of log aggregation, it is very hard to be able to reproduce the actions performed by privileged users or to assess suspicious activity.Read more...