Zimbra

Zimbra Tips : How To Enable SPF Checking for Incoming Connection

Posted by

Usually, i am configure spf to my server for outgoing purpose. The spf records are defined in public dns use txt records. But, how to enable spf checking if there connection to my server?

The following is step by step how to enable spf checking for incoming connection.

You need to enable cbpolicyd as in the following guides : https://imanudin.net/2014/09/08/how-to-install-policyd-on-zimbra-8-5/. After enable policyd, please open policyd webui (http://IPZIMBRA:7780/webui/index.php) and create some groups, policy and spf.

# Create Groups

Select Policies | Groups. Select action and add groups. given name list_domain. On comment, you can empty or filled with comment. Select a group that has been made. On action, select members and fill with your domain. See the following example. make sure disabled status is no at groups or members groups

policyd-groups

# Create Policy

Select Policies | Main. Add new policy and give name or information like the following picture. Then submit query

policy-spf

select new policy has been made and select members on action. Add member and fill on source/destination with group that has been made. See the following example

policy-spf-members

Above configuration only check spf if email connection come from external domain (Gmail, Yahoo and etc) to my internal domain. If email connection come from internal domain to internal domain, or internal domain to external domain, spf checking will be ignore/skip. make sure disabled status is no

# Create SPF Check

Select SPF Checks | Configure. Select Add on Action and configure like follow. Then Submit

spf-configure

Make sure disabled status is no. Enable policyd checkspf and restart policyd service

su - zimbra
zmprov ms `zmhostname` zimbraCBPolicydCheckSPFEnabled TRUE
zmcbpolicydctl restart

SPF checking for incoming connection has been enabled and configured. Please see zimbra.log if getting spf fail.

The following is example when getting spf fail

Mar 10 18:45:43 smtp postfix/smtpd[28068]: NOQUEUE: reject: RCPT from c117-167.nanaonet.jp[119.18.167.117]: 554 5.7.1 <shaftssg@onet.pl>: Sender address rejected: Failed SPF check; Please see http://www.openspf.org/Why?s=mfrom;id=shaftssg%40onet.pl;ip=119.18.167.117;r=smtp.imanudin.net; onet.pl, Sender is not authorized by default to use 'shaftssg@onet.pl' in 'mfrom' identity (mechanism '-all' matched); from=<shaftssg@onet.pl> to=<xxxx@imanudin.net> proto=ESMTP helo=<[119.18.167.117]>

Good luck and hopefully useful 😀

75 comments

  1. Dear Ahmad. I’ve been testing Zimbra for a couple of days now and im looking at the server monitor that the outgoing message count is very high and currently the server only has two test users. How is this possible? Maybe my server is compromised?

    Thanks

    Reply

  2. Hello! You have great posts and I am glad to have found your page.

    I have a question about the above setup. Users that have outlook connected via Imap can not sent messages. They fail the SPF check even though they are authenticated via SMTP. error looks like this.

    Server error: ‘554 5.7.1 : Sender address
    rejected: Failed SPF check; Please see
    http://www.openspf.org/Why?s=mfrom;id=user%40example.com;ip=96.X.Y.Zip
    ;r=mail.example.net; example.com, Sender is not authorized by default to use
    ‘user@example.com’ in ‘mfrom’ identity (mechanism ‘-all’ matched)’

    What can I do to account for these users. Activate sync and web work with no issues.

    Reply

  3. Sorry for late reply. I did not get notification of your comment. The IP 96.x.y.z is actually the public IP from where the computer with outlook is sitting. Here is an example show more information.

    ## imap user showing home comcast IP when sending to another internal user ##
    Received-SPF: fail (example.com: Sender is not authorized by default to use ‘sanga.c@example.com’ in ‘mfrom’ identity (mechanism ‘-all’ matched)) receiver=mail.example.net; identity=mailfrom; envelope-from=”sanga.c@example.com”; helo=DESKTOP4OH085B; client-ip=75.74.180.166
    Received: from DESKTOP4OH085B (c-75-74-180-166.hsd1.fl.comcast.net [75.74.180.166])

    Basically when I have outlook configured with IMAP, the email going to internal address gets rejected. But going to external address like gmail.

    ## imap user showing correct mail server public IP when sending to gmail ##
    Received-SPF: pass (google.com: domain of sanga.c@example.com designates 74.299.135.46 as permitted sender) client-ip=74.299.135.46;
    Authentication-Results: mx.google.com;
    spf=pass (google.com: domain of sanga.c@example.com designates 74.299.135.46 as permitted sender) smtp.mailfrom=sanga.c@example.com

    Not sure if its my SPF setting or IMAP settings

    Reply

    1. Hello Sangamc,

      It’s seems strange to me. The Public IP should sent from your server instead of from your public IP that used to connect. Please check SMTP outgoing configuration on your Outlook

      Reply

  4. Not sure why the comment is not saving, but here goes again.

    The IP 96.x.y.z is showing as the public IP where the computer with outlook is connecting from. So for example if I am using outlook connected with IMAP from home, the IP that shows is my home Comcast internet IP

    Reply

  5. Ok i have figured out the issue. The main reason I wanted to enable SPF was to block people masquerading as accounting@mydomain.com (a mailbox that does not exist) and sending mail to my staff with cryptovirus.

    I enabled policyD SPF with one extra setting !%internal_ip in addition to !%list_domain.

    This blocked the fake mail as I expected but also blocked IMAP users from outlook since they match the !%internal_ip setting.

    What really needs to be done is:
    – Reject mail from false senders
    https://wiki.zimbra.com/wiki/Rejecting_false_%22mail_from%22_addresses#Zimbra_Collaboration_8.5_and_above

    &
    – Enforce match between from and SASL username
    https://wiki.zimbra.com/wiki/Enforcing_a_match_between_the_FROM_address_and_the_sasl_username

    Lastly
    – Configure SPF with only the !%list_domain option.

    Now All is working correctly.

    Reply

    1. Hello Amit,

      That’s mean, your SPF configuration already works and can trap external domain who not use proper public IP for sending email

      Reply

  7. hi iman ,
    thanks for your reply
    how can i add particular domain to pass spf to receive mail
    is there any way to get mails from those users

    thanks
    amith

    Reply

  10. Hi Iman ,
    for example hdfcbank.com mails are rejectes
    so i should create group under list_domain like @hdfcbank.com
    and by pass the mail right

    Amith

    Reply

  12. Finally got this working on ZCS 8.7
    1. Install fails, the sqlite database is missing two key tables. The script that imports the tsql files to populate /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb database does not work, the last line fails ie ./convert-tsql sqlite core.tsql > /tmp/core.sql.

    Had to replace the line above with this:
    grep -v “#” /tmp/core.sql | sqlite3 /opt/zimbra/data/cbpolicyd/db/cbpolicyd.sqlitedb

    Then import all the tsql files in /opt/zimbra/common/share/database
    The Zimbra Wiki details what tables should be n this database
    https://wiki.zimbra.com/wiki/How-to_for_cbpolicyd

    2. Still did not appear to be working, /opt/zimbra/log/cbpolicyd.log initialising but no traffic. Also needed this:
    zmprov ms $(zmhostname) zimbraMtaEnableSmtpdPolicyd TRUE

    Now can see QUOTA and CheckSPF processing in cbpolicyd log

    Thanks iman

    Regards
    Geoffrey

    Reply

  13. Hello Iman,

    good to see you in this wall after long time.

    I am on requirement to block the sender domain sending mail without MX record. can you help me on this..??

    Awaiting your response..!!

    Reply

  15. Hi Iman,
    I like your post – nice and very useful job. So my question is about starting services. When you configure policy,
    rule etc is it not equal with starting service? Do we have to start it form cmd line (..) zimbraCBPolicydCheckSPFEnabled TRUE (…).? So what about Amvis Integration? When we configure i.e. new policy with blacklist domain do we have to start something from cmd?

    Reply

  16. Hello I have a question, spf check block mails from domains if who using softfail (im using fail),
    but this domain using email from hosting pop3 and they dont know how to do all this..

    i can take them domains to trusted or whitelist for spf check ?
    if yes 🙂 How ?

    Reply

    1. Hi Ali Dogan,

      SPF checking by default will reject if have spf fail (hard fail). If you have another domain and want to whitelist, you can add your domain into list_domain on group policy

      Reply
  17. Pingback: Zimbra Tips : How To Enforce SPF Checking For Incoming Email – Ahmad Imanudin

  18. Mas butuh pencerahan nya,,kenapa email yg saya kirim dari mail server zimbra saya selalu dianggap spam oleh gmail maupun yahoo? padahal saya sudah setting PTR, DKIM, SPF, DMARC dengan benar,,saya tes lewat mxtool juga hasil nya pass semua,,di header email nya pun spf,,dkim,,dmarc status nya pass…saya cek lewat mail-tester juga dapat score 10/10…
    reputasi IP pun saya cek di talos hasil nya neutral…apakah ada yang kurang dari konfigurasi saya?mohon bimbingannya mas…terima kasih…

    Reply

        1. seperti nya saya ketemu sebabnya mas,,
          ada akun yang mengirim email ke alamat”email yahoo dan gmail,,ini yang bikin reputasi mail server saya jadi jelek,,yg ke gmail sudah tidak terjadi lagi,,kemarin setelah saya tambahkan spf,,dkim dan ptr untuk incoming mailnya,,nah sekarang gantian yg ke yahoo ada akun (beda dengan yang kirim ke gmail) yg mengirim email ke banyak alamat”email yahoo..padahal sepertinya tidak ada dari pemilik akun tersebut mengirimkan email”tersebut,,cara mengatasi yang menggunakan resource mail server saya ini gmn yaa mas?sulit untuk meningkatkan reputasi mail server saya kalo kyk gini terus,,email saya akan selalu dikenali sebagai spam,,

          Reply

          1. Hi mas,

            Coba lakukan beberapa improvement berikut :

            https://imanudin.net/2014/09/08/how-to-install-policyd-on-zimbra-8-5/
            https://imanudin.net/2014/09/09/zimbra-tips-how-to-configure-rate-limit-sending-message-on-policyd/
            https://imanudin.net/2014/09/11/improving-anti-spam-reject-unlisted-domain-on-zimbra-8-5/
            https://imanudin.net/2014/09/12/zimbra-tips-how-to-protect-policyd-webui/
            https://imanudin.net/2014/09/29/how-to-restrict-users-sending-to-certain-usersdomains-with-policyd/
            https://imanudin.net/2014/09/30/script-automatic-configure-cbpolicyd-on-zimbra-8-5/
            https://imanudin.net/2014/09/07/how-to-improvement-sender-must-loginenforcing-a-match-between-from-address-and-sasl-username-on-zimbra-8-5/

  19. Hi iman,
    I set up the SPF check, but all incoming mail was rejected, I checked the incoming mail, there was the SPF record. For example: gmail, qq, 163 and so have been rejected, which is why?

    Reply

  20. Hi iman
    zimbra.log:
    Jul 13 09:12:24 webmail postfix/smtpd[30761]: connect from mail.cyagen.net[172.16.251.254]
    Jul 13 09:12:24 webmail postfix/smtpd[18824]: NOQUEUE: reject: RCPT from mail.cyagen.net[172.16.251.254]: 554 5.7.1 : Sender address rejected: Failed SPF check; qq.com, Sender is not authorized by default to use ‘78123538@qq.com’ in ‘mfrom’ identity, however domain is not currently prepared for false failures (mechanism ‘~all’ matched); from= to= proto=ESMTP helo=
    Jul 13 09:12:24 webmail saslauthd[14833]: zmauth: authenticating against elected url ‘https://webmail.cyagen.net:7073/service/admin/soap/’ …
    Jul 13 09:12:24 webmail postfix/smtpd[18824]: disconnect from mail.cyagen.net[172.16.251.254] ehlo=2 starttls=1 mail=1 rcpt=0/1 quit=1 commands=5/6
    Jul 13 09:12:24 webmail saslauthd[14833]: zmpost: url=’https://webmail.cyagen.net:7073/service/admin/soap/’ returned buffer->data=’soap:Senderauthentication failed for [cangcuiman]account.AUTH_FAILEDqtp649734728-11790:1499908344438:6e45d5c677204ab7′, hti->error=”
    Jul 13 09:12:24 webmail saslauthd[14833]: auth_zimbra: cangcuiman auth failed: authentication failed for [cangcuiman]
    Jul 13 09:12:24 webmail saslauthd[14833]: do_auth : auth failure: [user=cangcuiman] [service=smtp] [realm=] [mech=zimbra] [reason=Unknown]
    Jul 13 09:12:24 webmail postfix/smtpd[30761]: warning: mail.cyagen.net[172.16.251.254]: SASL LOGIN authentication failed: authentication failure
    Jul 13 09:12:24 webmail postfix/smtpd[30761]: lost connection after AUTH from mail.cyagen.net[172.16.251.254]
    Jul 13 09:12:24 webmail postfix/smtpd[30761]: disconnect from mail.cyagen.net[172.16.251.254] ehlo=1 auth=0/1 commands=1/2
    Jul 13 09:12:24 webmail postfix/postscreen[17910]: CONNECT from [172.16.251.254]:52814 to [172.16.10.195]:25
    Jul 13 09:12:24 webmail postfix/postscreen[17910]: PASS OLD [172.16.251.254]:52814
    qq.com Bounce:
    host mail2.cyagen.net[223.112.80.227] said: 554 5.7.1 : Sender address rejected: Failed SPF check; qq.com, Sender is not authorized by default to use ‘78123538@qq.com’ in ‘mfrom’ identity, however domain is not currently prepared for fals
    Thanks iman

    Reply

          1. No, my domain is cyagen.net
            This information is QQ mailbox bounce information：

            host mail2.cyagen.net[223.112.80.227] said: 554 5.7.1 : Sender address rejected: Failed SPF check; qq.com, Sender is not authorized by default to use ‘78123538@qq.com’ in ‘mfrom’ identity, however domain is not currently prepared for fals

            Thanks

          2. Hi Ken,
            It’s so strange. Because qq.com use ~ instead of – on their SPF records. Could you check/make sure qq.com come from their IP? is there any information in the log about IP address of qq.com?

  21. Mas Iman,
    Saya sudah coba terapkan Check SPF incoming email.
    Dampaknya beberapa domain customer saya ikut terblokir padahal domain2 dengan reputasi baik seperti kawasaki.co.id, hino-motors.co.id, dan bbrapa domain lain.
    Apakah ada cara untuk me-white list domain2 tersebut agar tidak ikut kena Check SPF ?

    Terima kasih sebelumnya.

    Reply

  24. Salam Hormat Mas iman

    Mohon maaf bertanya
    Saya juga ada error di @kemendag.go.id kurang lebih spt ini:

    root@papandayan:/home/user# cat /var/log/mail/mail.log | grep inatrade@kemendag.go.id
    Oct 12 09:58:05 papandayan postfix/policy-spf[20220]: : SPF fail (Mechanism ‘-all’ matched): Envelope-from: inatrade@kemendag.go.id
    Oct 12 09:58:05 papandayan postfix/smtpd[19675]: NOQUEUE: reject: RCPT from unknown[180.250.76.102]: 550 5.7.1 : Recipient address rejected: Please see http://www.openspf.org/Why?s=mfrom;id=inatrade%40kemendag.go.id;ip=180.250.76.102;r=papandayan.mra.co.id; from= to= proto=ESMTP helo=
    root@papandayan:/home/user#

    Saya harus melakukan langkah apa mas ?
    agar email dari @kemendag tsb masuk
    Terima Kasih sebelumnya

    Salam

    Irwan.Y

    Reply

      1. Mas Iman

        Mohon maaf ini saya dari sisi penerima yg tidak bisa terima email dari email @kemendag saja problem.
        dan itu juga tidak semua email..ada beberapa saja yg errornya spt diatas.
        klo terima email dari yg lain Normal
        apakah caranya sama spt diatas yg ditunjukan Mas Iman?
        Maksudnya konfigurasinya di sisi saya (penerima) ya mas ?
        Terima Kasih

        Salam

        Reply

  25. I there iman and thanks for your guide.

    This is my scenario

    Zimbra 8.0.7 GA Ubuntu 14.04LTS with cbpolicyd 2.1

    Public IP: a.a.a.a
    MTA IP: b.b.b.b

    example.com SPF public TXT record: v=spf1 mx a ip4:b.b.b.b/32 ~all
    example2.com SPF public TXT record: v=spf1 mx a ip4:b.b.b.b/32 ~all

    SPF/DKIM/DMARC for domain example.com/example2.com is correctly configured and verified with gLock Apps & MXtoolbox, but when I send email to test it something I can’t understand is goin’ on…

    I send an email from example2.com (always on same Zimbra server and network) to example.com user using a standard IMAP client like thunderbird/outlook and I receive this softfail SPF error

    CBPOLICYD LOG (/opt/zimbra/log/cbpolicyd.log)

    Received-SPF: softfail (example2.com: Sender is not authorized by default to use ‘test@example2.com’ in ‘mfrom’ identity, however domain is not currently prepared for false failures (mechanism ‘~all’ matched)) receiver=mail.example.com; identity=mailfrom; envelope-from=”spf@example2.com”; helo=”[MY_PC_IP_ADDRESS]”; client-ip=a.a.a.a
    Received: from [MY_PC_IP_ADDRESS] (unknown [a.a.a.a])

    POSTFIX

    Oct 20 17:10:57 mail postfix/smtps/smtpd[6878]: NOQUEUE: filter: RCPT from unknown[a.a.a.a]: : Sender address triggers FILTER smtp-amavis:[127.0.0.1]:10026; from= to= proto=ESMTP helo=
    Oct 20 17:10:57 mail postfix/smtps/smtpd[6878]: 861C582264C: client=unknown[a.a.a.a], sasl_method=PLAIN, sasl_username=test@example2.com

    Keep in mind I’ve configured SMTP to use my MTA hostname (mail.example.com -> b.b.b.b) , nothing strange here, standard SSL connection on 465 tcp port.

    I was expecting my client to use MTA IP address (b.b.b.b) instead it connects through my network gateway IP address (a.a.a.a) and SPF doesn’t match DNS records because this ip is not listed (as it should be) and I got a softfail error…. but the question is, why SPF policyd check is searching for client-ip address tag instead MTA’ sending IP?

    This obviously don’t happens if i use webmail that send email from expected b.b.b.b IP address causing no errors on SPF matching.
    Any idea?

    Reply

    1. Hi Andrea B,
      Could you try to block port 25 outgoing from all ip except from your email server. I want to ensure all outgoing is come from email server

      Reply

  27. Hi Mas,
    Untuk mematikan fitur Check SPF bagaimana ya?
    Saya sudah terapkan Check SPF dan berjalan baik, tapi ternyata banyak domain dari pengadaan tender ikut terblok karena banyak yg tidak memiliki SPF sehingga cukup menggangggu jalannya bisnis, mohon pencerahannya.

    Tks sebelumnya.

    Reply

    1. Hi mas Addo,
      Bisa dengan cara mengubah rule pada SPF dari disable=no menjadi disable=yes. Setelah itu, restart service cbpolicyd nya

      Reply

  28. Hi Iman,
    I executed the zmprov ms `zmhostname` zimbraCBPolicydCheckSPFEnabled TRUE statement, then I executed the zmcbpolicydctl restart statement so the server could not start the policyd service. What should I do?
    I configured on MTA Server.

    Thank you.

    Reply

    1. Hi Kaidou,
      Please see info on /opt/zimbra/log/cbpolicyd.log and /var/log/zimbra.log to gather information why your Policyd did not starting

      Reply

  29. ini berlaku untuk orang ngirim dari bad ip aja ya pak?

    kalo dia relay dari smtp relay yang rep nya bagus misal sendgrid, sendinblue masih bisa masuk yah ? walo dia nyamar jadi @domainkita

    Reply

    1. Hi mas,
      Rule ini berlaku untuk semua IP (good or bad). Jika email dikirimkan dari IP pub yang tidak terdaftar di SPF, maka akan di tolak

      Reply
  30. Pingback: How To Enable SPF Checking for Incoming Connection | All About Information Technology

  31. Hi Imaudin,

    First of all thank you for the amazing tutorial can this configure on zimbra 8.0.

    Im getting below error when i try to enable the zimbraCBPolicydCheckSPFEnabled

    [zimbra@mail ~]$ zmprov ms `zmhostname` zimbraCBPolicydCheckSPFEnabled TRUE
    ERROR: account.INVALID_ATTR_NAME (invalid attr name: invalid attr name – unable to modify attributes: zimbraCBPolicydCheckSPFEnabled: attribute type undefined)

    Reply

    1. Hello Manoj,
      You can use zmlocalconfig instead of zmprov. Please check with this command :

      su - zimbra
zmlocalconfig -s | grep -i policyd
      Reply

  32. Dear Brother Assalamu Alaikum,

    How are you I think all are well. Could you please help me…? In zimbra multi server when I create user1 and user2 it’s auto switch mailbox mx1 and mx2 simultaneously. The problem is when I login user2 using proxy, all are fine but when user1 login “Mail Folder” of left bar are not shown only zimlet are shown, as if it’s created on mx1.
    Please help me what’s the problem.

    Reply

    1. Waalaikumussalam,
      Please run this command on your mailbox server

      su - zimbra
/opt/zimbra/libexec/zmproxyconfig -e -w -m -H mbox1.imanudin.net
      Reply

      1. Dear Brother Assalamu Alaikum,
        Thanks for your reply. I have all 2 server such as ldap1, ldap2,mx1,mx2,mta1,mta2 and proxy1 proxy2. I run this command both proxy server /opt/zimbra/libexec/zmproxyconfig -e -w -m -H -r `hostname` but problem remain same. please help me.

        Reply

  33. i have tested your spf config and when email is send that does not have spf it is not rejected, but in web i have done reject, what could be wrong?

    Reply

  35. Hi, Imanudin.
    How can i add an exception to not check spf in a multidomain server for mails with from: and to: in internal domains?

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.