Showing results for 
Search instead for 
Do you mean 
Reply
Centrify
Centrify
Fel
Posts: 638
Registered: ‎07-06-2010

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

On a system experiencing the behavior, edit the /etc/centrifydc/ssh/sshd_config and set the LogLevel to Debug 3.

 

Restart SSHD and you should not get more verbose logging for SSH.  Leave Centrify Debug on as well and next time the issue occurs send the log files as previously indicated.

Felderi Santiago
Technical Manager - LATAM
Centrify Corporation
Found my response helpful? Click the Kudos button!
Please use plain text.
Frequent Advisor
wingZero21
Posts: 67
Registered: ‎11-14-2012

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi,

 

I have added the below showing the debug out for the fail and when it allowed access.

 

I had to restart the centrifydc service on this occasion to allow login via AD. Will add the adinfo -t as well.

 

During FAILURE

 

PAM: Authentication failure for illegal user mark.bell from ****-l15496.****.****.co.uk

debug3: mm_request_send entering: type 55 [MONITOR_ANS_PAM_RESPOND]

debug3: mm_sshpam_query: pam_query returned -1 [preauth]

debug2: auth2_challenge_start: devices <empty> [preauth]

debug3: mm_sshpam_free_ctx [preauth]

debug3: mm_request_send entering: type 58 [MONITOR_REQ_AUDIT_EVENT] [preauth]

debug3: mm_sshpam_free_ctx: waiting for MONITOR_ANS_PAM_FREE_CTX [preauth]

debug3: mm_request_receive_expect entering: type 59 [MONITOR_REQ_AUDIT_COMMAND] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 58 [MONITOR_REQ_AUDIT_EVENT]

debug3: mm_answer_pam_free_ctx

debug3: PAM: sshpam_free_ctx entering

debug3: PAM: sshpam_thread_cleanup entering

debug3: mm_request_send entering: type 59 [MONITOR_REQ_AUDIT_COMMAND]

debug2: monitor_read: 58 used once, disabling now

Failed keyboard-interactive/pam for invalid user mark.bell from 192.168.101.210 port 57572 ssh2

debug3: Received SSH2_MSG_IGNORE [preauth]

debug1: userauth-request for user mark.bell service ssh-connection method keyboard-interactive [preauth]

debug1: attempt 3 failures 2 [preauth]

debug2: input_userauth_request: try method keyboard-interactive [preauth]

debug1: keyboard-interactive devs  [preauth]

debug1: auth2_challenge: user=mark.bell devs= [preauth]

debug1: kbdint_alloc: devices 'pam' [preauth]

debug2: auth2_challenge_start: devices pam [preauth]

debug2: kbdint_next_device: devices <empty> [preauth]

debug1: auth2_challenge_start: trying authentication method 'pam' [preauth]

debug3: mm_sshpam_init_ctx [preauth]

debug3: mm_request_send entering: type 52 [MONITOR_REQ_PAM_QUERY] [preauth]

debug3: mm_sshpam_init_ctx: waiting for MONITOR_ANS_PAM_INIT_CTX [preauth]

debug3: mm_request_receive_expect entering: type 53 [MONITOR_ANS_PAM_QUERY] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 52 [MONITOR_REQ_PAM_QUERY]

debug3: mm_answer_pam_init_ctx

debug3: PAM: sshpam_init_ctx entering

debug3: PAM: sshpam_thread_conv entering, 1 messages

debug3: ssh_msg_send: type 1

debug3: ssh_msg_recv entering

debug3: mm_request_send entering: type 53 [MONITOR_ANS_PAM_QUERY]

debug3: mm_sshpam_query [preauth]

debug3: mm_request_send entering: type 54 [MONITOR_REQ_PAM_RESPOND] [preauth]

debug3: mm_sshpam_query: waiting for MONITOR_ANS_PAM_QUERY [preauth]

debug3: mm_request_receive_expect entering: type 55 [MONITOR_ANS_PAM_RESPOND] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 54 [MONITOR_REQ_PAM_RESPOND]

debug3: mm_answer_pam_query

debug3: PAM: sshpam_query entering

debug3: ssh_msg_recv entering

debug3: mm_request_send entering: type 55 [MONITOR_ANS_PAM_RESPOND]

debug3: mm_sshpam_query: pam_query returned 0 [preauth]

Postponed keyboard-interactive for invalid user mark.bell from 192.168.101.210 port 57572 ssh2 [preauth]

PAM: Authentication failure for illegal user mark.bell from ****-l15496.****.****.co.uk

debug3: mm_request_send entering: type 55 [MONITOR_ANS_PAM_RESPOND]

debug3: mm_sshpam_query: pam_query returned -1 [preauth]

debug2: auth2_challenge_start: devices <empty> [preauth]

debug3: mm_sshpam_free_ctx [preauth]

debug3: mm_request_send entering: type 58 [MONITOR_REQ_AUDIT_EVENT] [preauth]

debug3: mm_sshpam_free_ctx: waiting for MONITOR_ANS_PAM_FREE_CTX [preauth]

debug3: mm_request_receive_expect entering: type 59 [MONITOR_REQ_AUDIT_COMMAND] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 58 [MONITOR_REQ_AUDIT_EVENT]

debug3: mm_answer_pam_free_ctx

debug3: PAM: sshpam_free_ctx entering

debug3: PAM: sshpam_thread_cleanup entering

debug3: mm_request_send entering: type 59 [MONITOR_REQ_AUDIT_COMMAND]

debug2: monitor_read: 58 used once, disabling now

Failed keyboard-interactive/pam for invalid user mark.bell from 192.168.101.210 port 57572 ssh2

debug3: Received SSH2_MSG_IGNORE [preauth]

debug1: userauth-request for user mark.bell service ssh-connection method keyboard-interactive [preauth]

debug1: attempt 4 failures 3 [preauth]

debug2: input_userauth_request: try method keyboard-interactive [preauth]

debug1: keyboard-interactive devs  [preauth]

debug1: auth2_challenge: user=mark.bell devs= [preauth]

debug1: kbdint_alloc: devices 'pam' [preauth]

debug2: auth2_challenge_start: devices pam [preauth]

debug2: kbdint_next_device: devices <empty> [preauth]

debug1: auth2_challenge_start: trying authentication method 'pam' [preauth]

debug3: mm_sshpam_init_ctx [preauth]

debug3: mm_request_send entering: type 52 [MONITOR_REQ_PAM_QUERY] [preauth]

debug3: mm_sshpam_init_ctx: waiting for MONITOR_ANS_PAM_INIT_CTX [preauth]

debug3: mm_request_receive_expect entering: type 53 [MONITOR_ANS_PAM_QUERY] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 52 [MONITOR_REQ_PAM_QUERY]

debug3: mm_answer_pam_init_ctx

debug3: PAM: sshpam_init_ctx entering

debug3: PAM: sshpam_thread_conv entering, 1 messages

debug3: ssh_msg_send: type 1

debug3: ssh_msg_recv entering

debug3: mm_request_send entering: type 53 [MONITOR_ANS_PAM_QUERY]

debug3: mm_sshpam_query [preauth]

debug3: mm_request_send entering: type 54 [MONITOR_REQ_PAM_RESPOND] [preauth]

debug3: mm_sshpam_query: waiting for MONITOR_ANS_PAM_QUERY [preauth]

debug3: mm_request_receive_expect entering: type 55 [MONITOR_ANS_PAM_RESPOND] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 54 [MONITOR_REQ_PAM_RESPOND]

debug3: mm_answer_pam_query

debug3: PAM: sshpam_query entering

debug3: ssh_msg_recv entering

debug3: mm_request_send entering: type 55 [MONITOR_ANS_PAM_RESPOND]

debug3: mm_sshpam_query: pam_query returned 0 [preauth]

Postponed keyboard-interactive for invalid user mark.bell from 192.168.101.210 port 57572 ssh2 [preauth]

 

After restarting Centrifydc service

Postponed keyboard-interactive/pam for mark.bell from 192.168.101.210 port 57585 ssh2 [preauth]

debug3: Received SSH2_MSG_IGNORE [preauth]

debug3: mm_sshpam_respond [preauth]

debug3: mm_request_send entering: type 56 [MONITOR_REQ_PAM_FREE_CTX] [preauth]

debug3: mm_sshpam_respond: waiting for MONITOR_ANS_PAM_RESPOND [preauth]

debug3: mm_request_receive_expect entering: type 57 [MONITOR_ANS_PAM_FREE_CTX] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 56 [MONITOR_REQ_PAM_FREE_CTX]

debug3: mm_answer_pam_respond

debug2: PAM: sshpam_respond entering, 0 responses

debug3: mm_request_send entering: type 57 [MONITOR_ANS_PAM_FREE_CTX]

debug3: mm_sshpam_respond: pam_respond returned 0 [preauth]

debug3: mm_sshpam_free_ctx [preauth]

debug3: mm_request_send entering: type 58 [MONITOR_REQ_AUDIT_EVENT] [preauth]

debug3: mm_sshpam_free_ctx: waiting for MONITOR_ANS_PAM_FREE_CTX [preauth]

debug3: mm_request_receive_expect entering: type 59 [MONITOR_REQ_AUDIT_COMMAND] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 58 [MONITOR_REQ_AUDIT_EVENT]

debug3: mm_answer_pam_free_ctx

debug3: PAM: sshpam_free_ctx entering

debug3: PAM: sshpam_thread_cleanup entering

debug3: mm_request_send entering: type 59 [MONITOR_REQ_AUDIT_COMMAND]

debug2: monitor_read: 58 used once, disabling now

debug3: mm_request_receive_expect entering: type 50 [MONITOR_REQ_PAM_INIT_CTX]

debug3: mm_request_receive entering

debug1: do_pam_account: called

debug3: mm_request_send entering: type 51 [MONITOR_ANS_PAM_INIT_CTX]

Accepted keyboard-interactive/pam for mark.bell from 192.168.101.210 port 57585 ssh2

debug3: mm_do_pam_account entering [preauth]

debug3: mm_request_send entering: type 50 [MONITOR_REQ_PAM_INIT_CTX] [preauth]

debug3: mm_request_receive_expect entering: type 51 [MONITOR_ANS_PAM_INIT_CTX] [preauth]

debug3: mm_request_receive entering [preauth]

debug3: mm_do_pam_account returning 1 [preauth]

debug3: mm_send_keystate: Sending new keys: 0x1ba22c00 0x1ba22910 [preauth]

debug3: mm_newkeys_to_blob: converting 0x1ba22c00 [preauth]

debug3: mm_newkeys_to_blob: converting 0x1ba22910 [preauth]

debug3: mm_send_keystate: New keys have been sent [preauth]

debug3: mm_send_keystate: Sending compression state [preauth]

debug3: mm_request_send entering: type 24 [MONITOR_REQ_KEYEXPORT] [preauth]

debug3: mm_send_keystate: Finished sending state [preauth]

debug1: monitor_read_log: child log fd closed

debug1: monitor_child_preauth: mark.bell has been authenticated by privileged process

debug3: mm_get_keystate: Waiting for new keys

debug3: mm_request_receive_expect entering: type 24 [MONITOR_REQ_KEYEXPORT]

debug3: mm_request_receive entering

debug3: mm_newkeys_from_blob: 0x1ba2b8b0(143)

debug2: mac_setup: found hmac-sha1

debug3: mm_get_keystate: Waiting for second key

debug3: mm_newkeys_from_blob: 0x1ba2b8b0(143)

debug2: mac_setup: found hmac-sha1

debug3: mm_get_keystate: Getting compression state

debug3: mm_get_keystate: Getting Network I/O buffers

debug3: mm_share_sync: Share sync

debug3: mm_share_sync: Share sync end

debug3: temporarily_use_uid getpwnam(mark.bell)

debug3: getpwnam returned 1967185138, uid = 1967185138

debug1: temporarily_use_uid: 1967185138/1967185138 (e=0/0)

debug1: ssh_gssapi_storecreds: Not a GSSAPI mechanism

debug1: restore_uid: 0/0

debug1: PAM: establishing credentials

debug3: PAM: opening session

User child is on pid 28778

debug1: PAM: establishing credentials

debug3: permantently_set_uid getpwnam(mark.bell)

debug3: getpwnam returned 1967185138, uid = 1967185138

debug1: permanently_set_uid: 1967185138/1967185138

debug2: set_newkeys: mode 0

debug2: cipher_init: set keylen (16 -> 32)

debug2: set_newkeys: mode 1

debug2: cipher_init: set keylen (16 -> 32)

debug1: Entering interactive session for SSH2.

debug2: fd 6 setting O_NONBLOCK

debug2: fd 7 setting O_NONBLOCK

debug1: server_init_dispatch_20

debug3: Received SSH2_MSG_IGNORE

debug1: server_input_channel_open: ctype session rchan 256 win 16384 max 16384

debug1: input_session_request

debug1: channel 0: new [server-session]

debug2: session_new: allocate (allocated 0 max 10)

debug3: session_unused: session id 0 unused

debug1: session_new: session 0

debug1: session_open: channel 0

debug1: session_open: session 0: link with channel 0

debug1: server_input_channel_open: confirm session

debug1: server_input_channel_req: channel 0 request pty-req reply 1

debug1: session_by_channel: session 0 channel 0

debug1: session_input_channel_req: session 0 req pty-req

debug1: Allocating pty.

debug3: mm_request_send entering: type 25 [MONITOR_REQ_PTY]

debug3: mm_request_receive entering

debug3: monitor_read: checking request 25 [MONITOR_REQ_PTY]

debug3: mm_answer_pty entering

debug2: session_new: allocate (allocated 0 max 10)

debug3: session_unused: session id 0 unused

debug1: session_new: session 0

lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory

lastlog_openseek: Couldn't stat /var/log/lastlog: No such file or directory

debug3: mm_request_send entering: type 26 [MONITOR_ANS_PTY]

debug3: mm_answer_pty: tty /dev/pts/1 ptyfd 5

debug3: mm_pty_allocate: waiting for MONITOR_ANS_PTY

debug3: mm_request_receive_expect entering: type 26 [MONITOR_ANS_PTY]

debug3: mm_request_receive entering

debug1: session_pty_req: session 0 alloc /dev/pts/1

debug1: server_input_channel_req: channel 0 request shell reply 1

debug1: session_by_channel: session 0 channel 0

debug1: session_input_channel_req: session 0 req shell

debug3: do_service_auth is disabled, user(mark.bell) service(dzssh-shell)

debug1: do_service_auth user(mark.bell) service(dzssh-shell) rc(0)

debug1: Setting controlling tty using TIOCSCTTY.

debug2: fd 3 setting TCP_NODELAY

debug2: channel 0: rfd 11 isatty

debug2: fd 11 setting O_NONBLOCK

debug3: fd 9 is O_NONBLOCK

Please use plain text.
Frequent Advisor
wingZero21
Posts: 67
Registered: ‎11-14-2012

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi,

 

I have mailed the files as requested.

 

Please use plain text.
Frequent Advisor
wingZero21
Posts: 67
Registered: ‎11-14-2012

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi,

 

Files have been mailed. Do you need any other information for this.

Please use plain text.
Centrify
Ilau
Posts: 201
Registered: ‎06-29-2010

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

[ Edited ]

Hi

 

This is because nsswitch.conf was modified by Puppet and erased the configuration to retrieve user information from our adclient:

 

===

 

/etc/nsswitch.conf:
# Generated by Puppet Master
passwd:     files [SUCCESS=return] ldap
shadow:     files [SUCCESS=return] ldap
group:      files [SUCCESS=return] ldap
automount:  files [SUCCESS=return] ldap

 

===

 

This leads to no NSS request has been passed to adclient and it only went through files or ldap service, therefore no AD user information was returned. This is also why sshd reported invalid user:

 

===

 

May 16 08:27:09 qaapp237 sshd[27785]: input_userauth_request: invalid user mark.bell [preauth]

 

===

 

Once restarted our adclient, adclient by default will check and reconfigure the nsswitch.conf if our module is not in place:

 


passwd: centrifydc      files
shadow: centrifydc      files
group: centrifydc       files

 

 

 

In order to resolve this issue, you will have to disable the Puppet Master from modifying the nsswitch.conf or make sure our service module are still in place if you allow Puppet Master to edit nsswtich.conf.

 

Thanks,

Ian

 

 

 

Please use plain text.
Frequent Advisor
wingZero21
Posts: 67
Registered: ‎11-14-2012

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi,

 

Thanks for the reply that seems to indicate what the problem is.

 

I have also noticed centrify changes the krb5.conf file and pam.d/system-auth file.

 

Are there any other files that get amended by centrify when running.

Please use plain text.
Centrify
Ilau
Posts: 201
Registered: ‎06-29-2010

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi ,

 

By default, adclient will configure the following files after "adjoined":

 

/etc/krb5.conf

/etc/nsswitch.conf

/etc/pam.d/* or /etc/pam.conf

 

And you can turn off the auto-configuration throught the following parameters in /etc/centrifydc/centrifydc.conf:

 

adclient.autoedit.nss: false

adclient.autoedit.pam: false

adclient.krb5.autoedit: false

 

Please run adreload after modified the files.

 

Hope this helps.

 

Thanks,

Ian

Please use plain text.
Frequent Advisor
wingZero21
Posts: 67
Registered: ‎11-14-2012

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi,

 

Thanks again for the info.

 

If those files are turned off for auto configuration, will centrify still be able to allow AD login?

Please use plain text.
Centrify
Ilau
Posts: 201
Registered: ‎06-29-2010

Re: Issue with ssh login using centrify occasionally and when auto installing with puppet

Hi

 

No, authentication for AD user does not work if our configuration is not in place.

These parameters provide flexibility for users who will need customization on the authentication order or any special configuration.

Unless you are familiar with those files, otherwise it would be better if you could leave adclient to do the configuration automatically.

 

Thanks,

Ian

 

Please use plain text.