SECRET-SEN CyberArk Sentry - Secrets Manager Questions and Answers

 Dual accounts is a password management method that uses two accounts with identical privileges to access a system, database, or application. One account is active and the other is inactive at any given time. The active account remains untouched during password rotation, while the inactive account has its password changed after a grace period. This way, the application can always use the active account without experiencing any delays or errors due to password expiration or change. The advantage of using dual accounts is that it ensures business continuity and seamless access to the target resource, especially for high load and critical applications. References: Manage Dual Accounts, Configure dual accounts

The CCP (Central Credential Provider) is a tool that enables applications to securely retrieve credentials from CyberArk Secrets Manager without hard-coding or storing them in files. The CCP can be installed on a single server or on multiple servers behind a load balancer for high availability and scalability. The load balancer is a device or service that distributes the network traffic among the CCP servers based on predefined rules and criteria.

The CCP supports multiple methods to authenticate applications, such as Allowed Machines, Client Certificate, OS User, Path, and Hash. These methods are based on registering information in the Vault with the unique application ID. For more information about the supported authentication methods, see Application authentication methods1.

When installing the CCP and configuring it for use behind a load balancer, some authentication methods may be affected by the load balancer’s behavior and settings. Specifically, the following authentication methods may be affected:

• Allowed Machines authentication: This method authenticates applications based on their IP address or hostname. If the load balancer replaces the source IP or hostname of the routed packets with its own IP or hostname, the CCP will not be able to authenticate the application that initiated the credential request. To enable the CCP to resolve the IP or hostname of the application, the load balancer needs to be configured as a transparent proxy or to attach the X-Forwarded-For header to the routed packets. For more information, see Load balance the Central Credential Provider2.
• Client Certificate authentication: This method authenticates applications based on their client certificate that is signed by a trusted certificate authority (CA). The client certificate is used to establish a secure and trusted connection between the application and the CCP. If the load balancer terminates the SSL connection before proxying the traffic to the CCP, the CCP will not be able to verify the client certificate of the application. To enable the CCP to validate the client certificate, the load balancer needs to be configured as a pass-through proxy or to forward the client certificate to the CCP. For more information, see Load balance the Central Credential Provider2.

The other authentication methods are not affected by the load balancer, as they do not rely on the IP, hostname, or certificate of the application. For example, the OS User method authenticates applications based on their Windows domain user, the Path method authenticates applications based on their URL path, and the Hash method authenticates applications based on a hash value that is generated from the application ID and a shared secret. These methods do not require any special configuration on the load balancer or the CCP.

The cause of the issue is A. The certificate based on the Follower DNS name is not present on the Leader. This means that the Leader does not have a certificate file that matches the Follower DNS name used in the seed request, and therefore cannot generate a valid seed file for the Follower. This results in a 500 error in the Seed Fetcher container log. To resolve the issue, you need to import a certificate with the Follower DNS name as the subject alt name on the Leader, and create a copy of the certificate file with a name that matches the Follower DNS name used in the seed request1.

This is the most likely cause of this issue because it creates a discrepancy between the passwords stored in the Primary Vault and the DR Vault, which affects the Vault Conjur Synchronizer service (Synchronizer) and the application. The Synchronizer is a service that synchronizes secrets from the CyberArk Vault to the Conjur database. The application is a client that retrieves secrets from the Conjur database using the Conjur REST API. The CPM is a component that manages the lifecycle of the passwords stored in the CyberArk Vault, such as changing, verifying, and reconciling them. If the CPM is writing password changes to the Primary Vault while the Synchronizer is configured to replicate from the DR Vault, the following scenario may occur:

• The CPM changes the password for an account in the Primary Vault and updates the password value in the Vault database.
• The Synchronizer does not detect the password change in the DR Vault, as the DR Vault database has not been updated yet with the new password value.
• The Synchronizer does not sync the new password value to the Conjur database, as it assumes that the password value in the DR Vault database is the latest and correct one.
• The application requests the password value from the Conjur database and receives the old password value, which is different from the new password value in the Primary Vault database.
• The application tries to use the old password value to access the target platform or device and fails, as the target platform or device expects the new password value.

This answer is based on the CyberArk Secrets Manager documentation1 and the CyberArk Secrets Manager training course2.

= Summon is a command-line tool that provides on-demand secrets access for common DevOps tools. It reads a file in secrets.yml format and injects secrets as environment variables into any process. The secrets.yml file is where you define which secrets to retrieve from a trusted store, such as CyberArk Secrets Manager. The secrets.yml file specifies the name and location of each secret, as well as the environment variable to assign it to. For example, a secrets.yml file could look like this:

DB_USERNAME: !var dev/my-app/db-username DB_PASSWORD: !var dev/my-app/db-password

This means that Summon will fetch the values of dev/my-app/db-username and dev/my-app/db-password from the trusted store, and assign them to the environment variables DB_USERNAME and DB_PASSWORD, respectively. Then, Summon will run the specified process with these environment variables set, and remove them once the process exits. This way, Summon enables secure and convenient access to secrets without exposing them in plain text or storing them in files.

References = Summon by cyberark - GitHub Pages; Using Summon to Manage Secrets as You Move From Dev to Prod

 Conjur uses TLS certificates for authentication between nodes and clients. These certificates are either self-signed by Conjur or imported from a third-party CA. All the certificates are stored in the /opt/conjur/etc/ssl directory from the Conjur containers. This directory contains the following files:

• ca.crt: The CA certificate used to verify all Conjur node certificates. This is either the self-signed Conjur CA certificate or the imported third-party CA certificate.
• server.crt: The server certificate used by the Conjur node for HTTPS and mTLS connections. This certificate contains the DNS names of the node and the load balancer in the CN and SAN fields.
• server.key: The private key corresponding to the server certificate.
• cert.pem: A symbolic link to the server certificate file.
• key.pem: A symbolic link to the server key file.

References: Certificate architecture, Certificate requirements, Rotate certificates

Learn more:

The correct process to upgrade the CCP Web Service is D. Uninstall and reinstall the CCP Web Service. The CCP Web Service is a component of the CyberArk Central Credential Provider (CCP) that enables applications to retrieve secrets from the CyberArk Vault using REST API calls. To upgrade the CCP Web Service, you need to first uninstall the existing CCP Web Service from the Windows Server Manager or the Control Panel, and then reinstall the CCP Web Service using the latest installation package from the CyberArk website. The installation package contains both the Credential Provider and the CCP Web Service components, and you need to run the AimWebService.msi file to install the CCP Web Service. You also need to make sure that the CCP Web Service has the correct configuration and permissions, and that the CyberArk CRL (Certificate Revocation List) is open from the CCP server.

The other options are not correct processes to upgrade the CCP Web Service. Running “sudo yum update aimprv” from the CLI is a command to update the Credential Provider on Linux, not the CCP Web Service on Windows. Double-clicking the Credential Provider installer executable and selecting upgrade is a process to upgrade the Credential Provider on Windows, not the CCP Web Service. Double-clicking the AimWebService.msi and selecting upgrade is not a valid option, as the CCP Web Service does not support an upgrade option, and you need to uninstall it first before reinstalling it. References =

• Upgrade the Central Credential Provider (CCP) - CyberArk, Section “Upgrade the Central Credential Provider (CCP)”
• Central Credential Provider web service configuration - CyberArk, Section “Central Credential Provider web service configuration”

In the event of a Leader failure, you can perform a manual failover to promote one of the Standbys to be the new Leader. The manual failover process consists of the following steps:

• Suspend replication for all Standbys and Followers and identify the best failover candidate. This step ensures that no data is lost or corrupted during the failover process. The best failover candidate is the Standby with the most advanced replication timeline, which means it has the most up-to-date data from the Leader.
• Promote the failover candidate to be the new Leader. This step changes the role of the failover candidate from a Standby to a Leader, and updates its configuration accordingly. The new Leader can now accept write requests from clients and replicate data to other nodes.
• Restore replication. This step re-establishes the replication connections between the new Leader and the other nodes, and rebases the replication of the other Standbys and Followers to the new Leader. This ensures that all nodes have the same data and are in sync with the new Leader.

References: The manual failover configuration steps are explained in detail in the Configure Manual Failover section of the CyberArk Conjur Enterprise documentation. The image in the question is taken from the same source.

The best way to enable debug for CCP is to edit the web.config file in the AIMWebService folder and change the value of the AIMWebServiceTrace parameter to 4, which is the verbose level. This will generate detailed logs in the AIMWSTrace.log file in the logs folder. The logs folder may need to be created manually and given the appropriate permissions for the IIS_IUSRS group. After changing the web.config file, the Windows Web Server (IIS) service needs to be restarted to apply the changes. This method is recommended by CyberArk Support and documented in the CyberArk Knowledge Base1.

Editing the basic_appprovider.conf file and changing the AIMWebServiceTrace value is not a valid option, as this parameter does not exist in this file. The basic_appprovider.conf file is used to configure the basic provider settings, such as the AppProviderVaultParmsFile, the AppProviderPort, and the AppProviderCacheMode. The AIMWebServiceTrace parameter is only found in the web.config file of the AIMWebService.

In the PVWA, going to the Applications tab, selecting the Application in question, and going to Options > Logging and choosing Debug is not a valid option, as this will only enable debug for the Application Identity Manager (AIM) component, not the CCP component. The AIM component is used to manage the application identities and their access to the Vault. The CCP component is used to provide secure retrieval of credentials from the Vault using web services. Enabling debug for AIM will generate logs in the APPconsole.log, APPtrace.log, and APPaudit.log files in the ApplicationPasswordProvider\Logs folder, but these logs will not help to troubleshoot the CCP authentication issues.

From the command line, running appprvmgr.exe update_config logging=debug is not a valid option, as this will only enable debug for the Application Provider Manager (APM) component, not the CCP component. The APM component is used to manage the configuration and operation of the providers, such as the basic provider, the LDAP provider, and the ENE provider. Running appprvmgr.exe update_config logging=debug will generate logs in the appprvmgr.log file in the ApplicationPasswordProvider\Logs folder, but these logs will not help to troubleshoot the CCP authentication issues. References:

• Enable Debugging and Gather Logs - Central Credential Provider1

Conjur is a secrets management solution that securely stores and manages secrets and credentials used by applications, DevOps tools, and other systems. Conjur provides a REST API that enables users to perform various operations on Conjur objects, such as secrets, policies, roles, and resources. The API endpoint for each Conjur object is composed of the base URL of the Conjur server, followed by the object type and identifier. For example, the API endpoint for a secret named db-password in the dev/my-app policy is:

https:// /secrets/dev/my-app/db-password

To discover secrets inside of Conjur, the API endpoint that can be used is Resources. Resources are Conjur objects that have permissions and annotations associated with them, such as secrets, hosts, groups, and layers. The Resources API endpoint allows users to list, search, and filter resources based on various criteria, such as kind, owner, policy, and annotation. For example, the following API request will return a list of all secrets owned by the user alice:

https:// /resources?kind=variable&owner=user:alice

The Resources API endpoint can help users to discover secrets inside of Conjur by providing information such as the name, ID, policy, owner, and annotations of each secret. Users can also use the Resources API endpoint to check the permissions and audit records of each secret, and to retrieve the secret value if they have the read permission.

References = Conjur API; Resources API; Secrets API

According to the CyberArk Sentry Secrets Manager documentation1, the steps to failback to the primary site after a manual failover to the disaster recovery site are as follows:

• On the DR site, stop the Conjur Leader node using the command docker stop .
• On the primary site, generate a seed for the new Leader node using the command evoke seed leader . This will create a file named .tar in the current directory.
• On the primary site, copy the Leader seed file to the new Leader server using the command scp .tar :.tar
• On the new Leader server, create a new container using the same name as the one you just stopped, and load the Leader seed file using the command docker run --name -d --restart=always -v /var/log/conjur:/var/log/conjur -v /opt/conjur/backup:/opt/conjur/backup -p "443:443" -p "5432:5432" -p "1999:1999" cyberark/conjur:latest seed fetch .tar
• On the new Leader server, configure the Conjur Leader node using the command evoke configure leader -h -p
• On the new Leader server, reconfigure the Vault Conjur Synchronizer to point to the new Conjur Leader using the command evoke vault sync set
• On the DR site, generate a seed for the new Standby node using the command evoke seed standby . This will create a file named .tar in the current directory.
• On the DR site, copy the Standby seed file to the new Standby server using the command scp .tar :.tar
• On the new Standby server, create a new container using the same name as the one you just stopped, and load the Standby seed file using the command docker run --name -d --restart=always -v /var/log/conjur:/var/log/conjur -v /opt/conjur/backup:/opt/conjur/backup -p "443:443" -p "5432:5432" -p "1999:1999" cyberark/conjur:latest seed fetch .tar
• On the new Standby server, re-enroll the node to the cluster using the command evoke cluster enroll

The other options are not correct, as they are either unnecessary or incorrect. Contacting CyberArk for a new license file is not required, as the license is valid for both sites. Reconfiguring the Vault Conjur Synchronizer to point to the new Conjur Leader is a step that should be done on the new Leader server, not on the DR site. Triggering autofailover to promote the Standby in Site A to Leader is not possible, as the Standby node is not aware of the manual failover and will not accept the promotion request.

This is the correct answer because the Conjur CLI is a tool that allows users to interact with the Conjur REST API from the command line. The Conjur CLI can be run on Windows, Red Hat Enterprise Linux, and macOS operating systems, as well as in Docker containers. The Conjur CLI can be installed using various methods, such as downloading the executable file, using a package manager, or pulling the Docker image. The Conjur CLI supports Conjur Enterprise 12.9 or later versions. This answer is based on the CyberArk Secrets Manager documentation1 and the CyberArk Secrets Manager training course2.

The other options are not true statements for the Conjur CLI. The Conjur CLI can be run from any machine that has network access to the Conjur server, not only from the Conjur Leader node. The Conjur Leader node is the node that performs read/write operations on the Conjur database and policy engine, and hosts the Conjur UI and API endpoints. The Conjur CLI is not required for working with the Conjur REST API, as users can also use other tools, such as curl, Postman, or web browsers, to send HTTP requests to the Conjur REST API. The Conjur CLI does implement the Conjur REST API for managing Conjur resources, such as roles, policies, secrets, and audit records. The Conjur CLI provides a set of commands that correspond to the Conjur REST API endpoints and allow users to perform various operations on the Conjur resources.

 = This error occurs when the Vault Conjur Synchronizer installation script tries to log in to the Vault using the admin user credentials, but the admin user is already logged in to other sessions. The Vault has a limit on the number of concurrent sessions per user, and the default value is one. Therefore, the installation script fails to authenticate the admin user and returns the error message: Forbidden Logon Token is Empty - Cannot logon Unauthorized. To remediate the issue, the admin user must log out of any other sessions before running the installation script, or increase the limit on the number of concurrent sessions per user in the Vault configuration file12. References: =

• Troubleshoot CyberArk Vault Synchronizer 1, Error: Forbidden Logon Token is Empty - Cannot logon Unauthorized
• Vault.ini File Parameters 2, ConcurrentSessionsPerUser

The exhibit shows a JSON object that contains the replication status of a database in a Secrets Manager cluster. Secrets Manager is a secrets management solution that securely stores and manages secrets and credentials used by applications, DevOps tools, and other systems. Secrets Manager can be deployed in a cluster mode, which consists of a Leader node and one or more Follower nodes. The Leader node is the primary node that handles all write operations and coordinates the replication of data to the Follower nodes. The Follower nodes are read-only nodes that replicate data from the Leader node and serve requests from clients and applications that need to retrieve secrets or perform other read-only operations.

To confirm that the Follower has a current copy of the database, you can compare the pgcurrentxlog_locationlocation from the Leader to the Follower you need to validate against. The pgcurrentxlog_locationlocation is a property that indicates the current position of the write-ahead log (WAL) in the database. The WAL is a mechanism that records all changes made to the database in a sequential log file, before they are applied to the actual data files. The WAL ensures the durability and consistency of the database in case of a crash or a power failure. The WAL also enables the replication of data from the Leader node to the Follower nodes, by streaming the WAL records to the Follower nodes and applying them to their local databases.

By comparing the pgcurrentxlog_locationlocation from the Leader to the Follower, you can determine how far behind the Follower is from the Leader in terms of the WAL records. If the pgcurrentxlog_locationlocation values are identical or very close, it means that the Follower has a current copy of the database, and that the replication is working properly. If the pgcurrentxlog_locationlocation values are different or far apart, it means that the Follower has an outdated copy of the database, and that there is a replication lag or a replication failure. In that case, you may need to troubleshoot the replication issue and resolve it as soon as possible.

References = Secrets Manager Cluster Installation; Secrets Manager Cluster Configuration; Write-Ahead Logging - PostgreSQL Documentation