Category: CentOS

Analysing mail log details – Exim log file flags a detailed view

Analysing mail log details – Exim log file flags a detailed view

How to analyse the mail log details – Exim?

To play with email issues we’ve a good knowledge about the log file and different log line flags.

We know the basics of exim MTA and its basic commands. We already discussed the topic in one of  our previous post “How to heck spamming on server“.

You will get more details about the spamming and the commands to analyse it. Here I’m explaining about the exim mail log. Exim mail log file is “/var/log/exim_mainlog“

From the email queue, we can analyse the log details of an email by using the command “exim -Mvl Message-ID”

Sample email log (exim -Mvl) output:


# exim -Mvl 1VlxUy-0001ka-9V

2013-11-28 02:03:00 Received from sfsffsfr@cfsfost6.vfsfsflehfsfting.com U=swfsfcar P=local S=1349 T="Welcome to Swift Carders-The Carders' Home"
2013-11-28 02:03:00 SMTP error from remote mail server after RCPT TO:: host gmail-smtp-in.l.google.com [17.xxx.xxx27]: 450-4.2.1 The user you are trying to contact is receiving mail too quickly.n450-4.2.1 Please resend your message at a later time. If the user is able ton450-4.2.1 receive mail at that time, your message will be delivered. For moren450-4.2.1 information, please visitn450 4.2.1 http://support.google.com/mail/bin/answer.py?answer=6592 sn7si36197219pab.341 - gsmtp

Log line flags – details

One line should be there in mail log for each messages that received, and for each successful, unsuccessful, and delayed delivery. These lines can readily be picked out by the distinctive two-character flags that immediately follow the timestamp. The flags are:

<=     message arrival 
=>     normal message delivery
->     additional address in same delivery
*>     delivery suppressed by -N
**     delivery failed; address bounced
==     delivery deferred; temporary problem

Summary of the field identifiers that are used in log lines are listed below:

A           authenticator name (and optional id)
C           SMTP confirmation on delivery
            command list for “no mail in SMTP session”
CV          certificate verification status
D           duration of “no mail in SMTP session”
DN          distinguished name from peer certificate
DT          on => lines: time taken for a delivery
F           sender address (on delivery lines)
H           host name and IP address
I           local interface used
id          message id for incoming message
P           on <= lines: protocol used                          on => and ** lines: return path
QT          on => lines: time spent on queue so far
            on “Completed” lines: time spent on queue
R           on <= lines: reference for local bounce                          on =>  ** and == lines: router name
S           size of message
ST          shadow transport name
T           on <= lines: message subject (topic)                          on => ** and == lines: transport name
U           local user or RFC 1413 identity
X           TLS cipher suite

Hope this will give you a close look on exim. Thanks.. 🙂

Related post

1, Find out the spam mailing script’s location
2, Remove all frozen emails from mail queue exim

Dump Mongo DB and move it to an S3 bucket.

Requirement: Need to create a script to create Mongo DB dump and move the dump to an AWS s3 bucket.

Prerequisites: SSH access to Mongo DB server, IAM user with AWS s3 full [or write] access, aws-cli on server, knowledge in Mongo commands for dump creation.

As we need to move the dump to an S3 bucket, first we need to configure IAM user. Then only we can move the dump to S3 bucket. To configure IAM, you need to install aws-cli tool on the machine.

The post Analysing mail log details – Exim log file flags a detailed view appeared first on Crybit.com.

How to check spamming on server which has Exim MTA?

Electronic spamming is the use of electronic messaging systems to send unsolicited messages (spam), especially advertising, as well as sending messages repeatedly on the same site.

It happens in many ways on a mail server. Spamming is one of the common and frequent problem in web hosting industry. Spamming can cause your server’s IPs blocked at different RBLs, if any accounts in your server sends multiple spam emails.

We can simply check and confirm whether the emails sending from accounts are real or spam by analyzing the email queue. In a cPanel server, the default MTA (Mail Transfer Agent) is Exim. Exim has different command line options to identify spam.

What is a Linux container?

An intro to Linux containerisation.

Basic: Containerisation helps to isolate processes. You can run your App/Services as an isolated process, running from a distinct image that provides all files necessary to support the processes.

Basically Linux containers are OS level virtualisation technique for running multiple isolated Linux systems (containers) on a control host using a single Linux kernel.

In most cases we will get the details from the email header itself, also we can check the body of that email and the email log by using different command-line options. After reading this article, you will get a clear idea on how to identify spamming on a server which uses Exim as MTA.

To check this, log into server as root.

To count emails in queue

exim -bpc

This command shows the total number of emails in the queue. If the count is high (>2000), the probability for spamming is also high.

Example

# exim -bpc
52

To list emails with more details

exim -bp

This command will give you a close look to the emails in the queue. It will give details like message ID, sender, recipient, size and age of mail. From this, the message ID is useful to find out the details like header, body and log. That will discussed in detail later.

Example

# exim -bp
44h 763 1VGaIo-0002ec-RM 
recipient@gmail.com

10h 5.9K 1VH6AW-0001Um-Rz <> *** frozen ***
no-reply@facebook.com

0m 502 1VHFNl-0003bf-GB 
recipient@gmail.com

0m 568 1VHFNl-0003bn-Tq 
recipient@gmail.com

# 1st field: Age (Eg : 44h)
# 2nd field: Size (Eg : 5.9K)
# 3rd field: Message ID (Eg : 1VGaIo-0002ec-RM)
# 4th field: Sender (Eg : sender@sender.com)
# 5th field: Recipient (Eg : recipient@gmail.com)

By using the ID we can analyse the header, body and the log information of emails in the queue.

exim -Mvh ID

This command displays the message header. From its output, we can check a lot of details about the email like; from address, to address, subject, date, script etc.

exim -Mvb ID

Displays the message body.

exim -Mvl ID

It displays the log of email. From this log, you can identify the user who created this emails and a lot more…

Spamming can occur in many ways. Here I am explaining some instances of spamming. It occurs mainly through vulnerable PHP scripts or by compromising the email account’s password.

Example: Spamming from PHP script

208P Received: from $user by server.ahostname.com with local (Exim 4.82)
(envelope-from <$user@server.ahostname.com>)
id 1YZUIE-00013s-Sp
for wend1122@yahoo.com; Sat, 21 Mar 2015 21:03:06 -0400
027T To: wend1122@yahoo.com
019 Subject: Hi there
091 X-PHP-Script: domain.com/templates/yoo_revista/warp/menus/page.php for "IP.Address"
023 X-Priority: 3 (Normal)

From the header itself, we can find out the email reputation. If you find “X-PHP-Script” in the email header, you can confirm that, those emails were sent out from a PHP script. In the above example the emails were sent from PHP script (X-PHP-Script: domain.com/templates/yoo_revista/warp/menus/page.php). In this case we have to check the scripts in the problematic account.

In this case, please make sure that you are using latest version of CMS (Eg; WordPress, Joomla etc), plugins and themes. 

Analyzing email count with sender

This’s very important while checking spamming. This command will sort out the email count with its sender name from the Exim mail queue. From this output we can analyse the email account who is sending large emails.

exim -bpr|grep "<"|awk {'print $4'}|cut -d"<" -f2|cut -d">" -f1|sort -n|uniq -c|sort -n

Example

See the example below:

[root@EcLinux]# exim -bpr|grep "<"|awk {'print $4'}|cut -d"<" -f2|cut -d">" -f1|sort -n|uniq -c|sort -n
3 sender@sender.com
1

Another way using “exiqgrep“

exiqgrep -f sendername|grep "<"|wc -l

This command displays the total count of emails that sent by a particular user.

Example

[root@EcLinux]# exiqgrep -f sender@sender.com|grep "<"|wc -l
3

Similarly -r switch with exiqgrep is using for recipient.

exiqgrep -f recipient|grep "<"|wc -l

Refer this for more details >> count emails in Exim mail queue for a specific sender/receiver <<

Removing emails from queue

The exim command to remove emails from queue is;

exim -Mrm

To delete all emails from queue for a particular sender.

exim -bpr| grep sendername| awk '{print $3}'|xargs exim -Mrm

The “awk” part prints the messages IDs to remove. This will give to the input of “exim -Mrm” by using xargs function.

To remove all emails from the queue, here is a quick solution >> remove all emails from the queue <<

Frozen emails

The sender field must have the word “frozen“. To displays the total count of frozen emails in queue, we can use the following command.

exim -bp|grep frozen|wc -l

Removing frozen emails

exim -bp|grep frozen|awk {'print $3'}|xargs exim -Mrm

We can simply remove all frozen emails from the queue by using the “exiqgrep” command. Please refer the following link >> Quick way to remove all frozen emails from the email queue <<

exim -bp|exiqsumm

The above command will print the summary of emails in queue.

Example

# exim -bp|exiqsumm
Count Volume Oldest Newest Domain
----- ------ ------ ------ ------
1 6041 11h 11h facebook.com
1 763 45h 45h interia.pl
---------------------------------------------------------------
2 6804 45h 11h TOTAL

exiwhat

It displays, what exim is doing right now. See the below example:

# exiwhat
1923 daemon: -q1h, listening for SMTP on port 25 (IPv6 and IPv4) port 587 (IPv6 and IPv4) and for SMTPS on port 465 (IPv6 and IPv4)

Related posts

1, Exim Log line flags
2, Command to find the mail that we have sent is completed or not!

The post How to check spamming on server which has Exim MTA? appeared first on Crybit.com.

CentOS Community Newsletter, August 2019 (#1908)

CentOS Community Newsletter, August 2019 (#1908)

Dear CentOS enthusiast,

It’s been another busy month, but better a few days late than never!

If you’d like to help out with the process of putting together the newsletter, please see the Contributing section at the end. We’re always looking for help!

Releases and updates

We had a very large number of updates/enhancements in July:

Errata and Enhancements Advisories

We issued the following CEEA (CentOS Errata and Enhancements Advisories) during July:

Errata and Security Advisories

We issued the following CESA (CentOS Errata and Security Advisories) during July:

Errata and Bugfix Advisories

We issued the following CEBA (CentOS Errata and Bugfix Advisories) during July:

Events

Last week we were at DevConf.in in Bangalore. If you dropped by, thanks!

Next week – August 14th – we’ll be gathering at Boston University, in Boston, Massachusetts, for the second annual CentOS Dojo at DevConf.US. There’s still space to register, if you wish to attend. In addition to the regular sessions, there will be an opportunity to give lightning talks about what you’re working on, as requested by last year’s attendees. More details are available on the event wiki page.

And the week after that – August 21-23 – we will be at the Open Source Summit in San Diego. Drop by to see us at the Red Hat booth!

If you are presenting anything about CentOS, at any event anywhere in the world, please do let us know, so that we can promote your presence there, and your talk.

If you’d like to run a CentOS Dojo, or other community event, we may be able to help. Get in touch via the centos-devel mailing list, or via our Twitter account @CentOSProject.

Contributing to CentOS Pulse

We are always on the look-out for people who are interested in helping to:

  • Tell us what you’re working on
  • Provide a report from the SIG on which you participate
  • Tell us about an event that you attended where there was CentOS content
  • Write an article on an interesting person or topic
  • Tell us about a news article that covered the use of CentOS in an interesting way
  • Suggest an topic that you’d like to see someone else write an article on

Please see the page with further information about contributing. You can also contact the Promotion SIG, or just email Rich directly (rbowen@centosproject.org) with ideas or articles that you’d like to see in the next newsletter.

 

CentOS Dojo at DevConf.US – August 14th, 2019 in Boston

The CentOS Project is pleased to be hosting a one-day Dojo, in conjunction with the upcoming DevConfUS conference, on August 14, 2019.

The one-day event, located on the campus of Boston University in the George Sherman Union Building, will feature talks on:

  • Running CentOS and Terraform on AWS
  • Supercomputing at NC State University
  • An Introduction to Keylime
  • Using Applications Streams
  • Lightning talks about what you’re working on

The event is free, but attendees should register for the event so planners can get an idea of attendance. 

In the evening we’ll be gathering at a local watering hole for less formal discussions, accompanied by food and great local beers – location to be announced on the day of the event!

CentOS will continue its presence at DevConfUS with a booth and various talks, so even if you miss the Dojo, there’s still plenty of time to meet with folks working on CentOS. We look forward to seeing you soon!

How to find IP Address that Launch DDOS Attack

If your VPS or server load suddenly increases much higher than normal, it could be a DDOS attack.

To find out which IPs did that do the following,

Option 1 :- If you know which domain is attacked. SSH to your server & issue the following command. Make sure you replace “DOMAIN” with your domain name. If you are using cPanel/WHM and the domain is not the primary domain, normally it will be the sub domain of the primary domain.

less /usr/local/apache/domlogs/DOMAIN | awk '{print $1}' | sort | uniq -c | sort -n

Option 2 :- If you don’t know which domain is attacked. SSH to your server & issue the following command. Option 1 if preferable especially if your server is very busy has many domain. It will take quite sometimes to process the log file. You can check by issuing “top -c” command to find out which domain consume the most resources.

less /usr/local/apache/logs/access_log | awk '{print $1}' | sort | uniq -c | sort -n

Both of the option will give the ip and number of connections in the descending order. For example:

.....
.....
.....
.....
17843 56.51.155.156
19234 66.156.66.266
234578 156.56.16.76

In the above case we can see too many connections from those ips and it is abnormal. You can block these ips in the firewall such as ConfigServer Firewall (“csf”).

CentOS Atomic Host 7.1906 Available for Download

The CentOS Atomic SIG has released an updated version of CentOS Atomic Host (7.1906), an operating system designed to run Linux containers, built from standard CentOS 7 RPMs, and tracking the component versions included in Red Hat Enterprise Linux Atomic Host.

CentOS Atomic Host includes these core component versions:

  • atomic-1.22.1-26.gitb507039.el7.centos.x86_64
  • rpm-ostree-client-2018.5-2.atomic.el7.x86_64
  • ostree-2018.5-1.el7.x86_64
  • cloud-init-18.2-1.el7.centos.2.x86_64
  • docker-1.13.1-96.gitb2f74b2.el7.centos.x86_64
  • kernel-3.10.0-957.21.3.el7.x86_64
  • podman-1.3.2-1.git14fdcd0.el7.centos.x86_64
  • flannel-0.7.1-4.el7.x86_64
  • etcd-3.3.11-2.el7.centos.x86_64

Download CentOS Atomic Host

CentOS Atomic Host is available as a VirtualBox or libvirt-formatted Vagrant box, or as an installable ISO, or qcow2 image. For links to media, see the CentOS wiki.

Upgrading

If you’re running a previous version of CentOS Atomic Host, you can upgrade to the current image by running the following command:

# atomic host upgrade

Release Cycle

The CentOS Atomic Host image follows the upstream Red Hat Enterprise Linux Atomic Host cadence. After sources are released, they’re rebuilt and included in new images. After the images are tested by the SIG and deemed ready, we announce them.

Getting Involved

CentOS Atomic Host is produced by the CentOS Atomic SIG, based on upstream work from Project Atomic. If you’d like to work on testing images, help with packaging, documentation – join us!

You’ll often find us in #atomic and/or #centos-devel if you have questions. You can also join the atomic-devel mailing list if you’d like to discuss the direction of Project Atomic, its components, or have other questions.

Getting Help

If you run into any problems with the images or components, feel free to ask on the centos-devel mailing list.

Have questions about using Atomic? See the atomic mailing list or find us in the #atomic channel on Freenode.

IBM, Red Hat, and CentOS

CentOS community,

Today marks a new day in the 26-year history of Red Hat. IBM has finalized its acquisition of Red Hat which will operate as a distinct unit within IBM moving forward.

What does this mean for Red Hat’s contributions to the CentOS project?

In short, nothing.

Red Hat always has and will continue to be a champion for open source and projects like CentOS. IBM is committed to Red Hat’s independence and role in open source software communities so that we can continue this work without interruption or changes.

Our mission, governance, and objectives remain the same. We will continue to execute the existing project roadmap. Red Hat associates will continue to contribute to the upstream in the same ways they have been. And, as always, we will continue to help upstream projects be successful and contribute to welcoming new members and maintaining the project.

We will do this together, with the community, as we always have.

If you have questions or would like to learn more about today’s news, I encourage you to review the list of materials below. Red Hat CTO Chris Wright will host an online Q&A session in the coming days where you can ask questions you may have about what the acquisition means for Red Hat and our involvement in open source communities. Details will be announced on the Red Hat blog

More info:

Press release

Chris Wright blog – Red Hat and IBM: Accelerating the adoption of open source

FAQ on Red Hat Community Blog

Updated CentOS Vagrant Images Available (v1905.01)

We are pleased to announce new official Vagrant images of CentOS Linux 6.10 and CentOS Linux 7.6.1810 for x86_64. All included packages have been updated to May 30th, 2019.

Known Issues

  1. The VirtualBox Guest Additions are not preinstalled; if you need them for shared folders, please install the vagrant-vbguest plugin and add the following line to your Vagrantfile: 
    config.vm.synced_folder ".", "/vagrant", type: "virtualbox"

    We recommend using NFS instead of VirtualBox shared folders if possible; you can also use the vagrant-sshfs plugin, which, unlike NFS, works on all operating systems.

  2. Since the Guest Additions are missing, our images are preconfigured to use rsync for synced folders. Windows users can either use SMB for synced folders, or disable the sync directory by adding the line 
    config.vm.synced_folder ".", "/vagrant", disabled: true

    to their Vagrantfile, to prevent errors on “vagrant up”.

  3. Installing open-vm-tools is not enough for enabling shared folders with Vagrant’s VMware provider. Please follow the detailed instructions in https://github.com/mvermaes/centos-vmware-tools
  4. Some people reported “could not resolve host” errors when running the centos/7 image for VirtualBox on Windows hosts. We don’t have access to any Windows computer, but some people reported that adding the following line to the Vagrantfile fixed the problem: 
    vb.customize ["modifyvm", :id, "--natdnshostresolver1", "off"]

Recommended Setup on the Host

Our automatic testing is running on a CentOS Linux 7 host, using Vagrant 1.9.4 with vagrant-libvirt and VirtualBox 5.1.20 (without the Guest Additions) as providers. We strongly recommend using the libvirt provider when stability is required.

Downloads

The official images can be downloaded from Vagrant Cloud. We provide images for HyperV, libvirt-kvm, VirtualBox and VMware.

If you never used our images before:

vagrant box add centos/6 # for CentOS Linux 6, or...
vagrant box add centos/7 # for CentOS Linux 7

Existing users can upgrade their images:

vagrant box update --box centos/6
vagrant box update --box centos/7

Verifying the integrity of the images

The SHA256 checksums of the images are signed with the CentOS 7 Official Signing Key. First, download and verify the checksum file:

$ curl http://cloud.centos.org/centos/7/vagrant/x86_64/images/sha256sum.txt.asc -o sha256sum.txt.asc
$ gpg --verify sha256sum.txt.asc

Once you are sure that the checksums are properly signed by the CentOS Project, you have to include them in your Vagrantfile (Vagrant unfortunately ignores the checksum provided from the command line). Here’s the relevant snippet from my own Vagrantfile, using v1803.01 and VirtualBox:

Vagrant.configure(2) do |config|
  config.vm.box = "centos/7"

  config.vm.provider :virtualbox do |virtualbox, override|
    virtualbox.memory = 1024
    override.vm.box_download_checksum_type = "sha256"
    override.vm.box_download_checksum = "b24c912b136d2aa9b7b94fc2689b2001c8d04280cf25983123e45b6a52693fb3"
    override.vm.box_url = "https://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-x86_64-Vagrant-1803_01.VirtualBox.box"
  end
end

Feedback

If you encounter any unexpected issues with the Vagrant images, feel free to ask on the centos-devel mailing list, or in #centos on Freenode IRC.

Ackowledgements

I would like to warmly thank Brian Stinson, Fabian Arrotin and Thomas Oulevey for their work on the build infrastructure, as well as Patrick Lang from Microsoft for testing and feedback on the Hyper-V images. I would also like to thank the CentOS Project Lead, Karanbir Singh, without whose years of continuous support we wouldn’t have had the Vagrant images in their present form.

I would also like to thank the following people (in alphabetical order):

  • Graham Mainwaring, for helping with tests and validations;
  • Michael Vermaes, for testing our official images, as well as for writing the detailed guide to using them with VMware Fusion Pro and VMware Workstation Pro;
  • Kirill Kalachev, for reporting and debugging the host name errors with VirtualBox on Windows hosts.