Linux Storage Management: Red5 Pro Case Study- by eliminating unnecessary files

For video streaming, we are using Red5 Pro software, hosted in an AWS EC2 instance. For the second time, we have encountered an issue out-of-storage in Linux. Since we notice the issue repeatedly, we investigate the reasons and also come to a permanent solution.

First, let me explain the application architecture and then I will explain, how we resolve the issue.

We had a feature to save the streaming video and we utilize a S3 bucket for this purpose.

We found out, that even though we store files in the S3 bucket, the Red5Pro uses the /tmp/live/streams directory to store these files. This causes the running out of storage issue.

This article is separated into two sections. In our very first section, we will figure out the issue. In the following section, we will discuss how we resolve this particular issue.

Find the folder/files/application for our Linux storage shortage

First, clean the cached packages from apt (Advance Package Tool)

<code>sudo apt-get clean</code>Code language: HTML, XML (xml)

Now remove packages that were installed as dependencies of other packages but are no longer required,

sudo apt-get autoremoveCode language: JavaScript (javascript)

Now time to investigate what is going on with our file storage. Let’s check our disk spaces,

df -h

With this query, we can check the disk usage, specially with the column named Use%.

Hopefully, from here, you can confirm you are running out of storage.

Now we need to dig deeper, to find out which application is responsible for this unfortunate situation. We will take the top 20 files/directories (According to your situation, you might need more than 20) of the file system.

First, go to the root directory,

cd /

Be a root/super-user,

sudo su

Now run the following command,

sudo du -ah . | sort -rh | head -20

This will be resulting the top 20 larger files/directories.

Now look closely and figure out which directory/files are taking unusual storage.

This is a way you will figure out the core reason for short storage.

Another example can be, we have a VS code running on the server. It took almost ~1 GB of storage. During the investigation, first we look for the directory,

df -h

It shows our /dev/xvda is using 89% of the total storage and we need to clean up the partition. To find out which directory or application is taking all this storage, from the root (/) directory we can simply run,

sudo du -ah . | sort -rh | head -20

It shows the cloud VS code server is using ~1GB of storage. Now In our case, the VS code was installed temporarily on the server and it’s no longer required. We now can delete the VS code server directory and restore the empty spaces.

Possible Solutions for storage issue

Here, we can see, that our Red5 Pro server is persisting files inside the /temp/live/streams directory. The fastest and easiest solution is to delete the contents of the directory.

But this is a temporary solution and in the future, it will store the video files again and we will encounter this short storage issue repeatedly. Instead, we could use one of the following scalable solutions,

• Using a tool called tmpreaper. This will remove the files that have not been accessed for certain days.
• Using CRON job to delete these files by ourselves within a specific period

Since the usage of tmpreaper is self-explanatory, we will discuss how we can make use of the CRON job.

To add a CRON job, first, you must install CRON. In Ubuntu, it is installed by default. If CRON is not available in your system, first install it and make sure the CRON service is up and running by,

systemctl status cron

In Ubuntu, the CRON job is written in /etc/crontab. So update the file with the relevant file deletion task. In my case, I have to delete the files from /tmp/live/streams/, when the files are 2 days older and this operation requires the super-user/root level permission.

In this particular case, instead of using /etc/crontab, we will utilize sudo crontab -e.

As a task in the cron job, I will use the find operation and delete files inside the /tmp/live/streams/.

We have to add the following line that will find all the files older than 2 days and delete them accordingly.

0 2 * * * /usr/bin/find /tmp/live/streams/ -type f -mtime +2 -exec rm -rf {} \;Code language: JavaScript (javascript)

First, open the file as root/super-user,

sudo crontab -e

Now add the line,

0 2 * * * /usr/bin/find /tmp/live/streams/ -type f -mtime +2 -exec rm -rf {} \;Code language: JavaScript (javascript)

Save the file and CRON service is smart to detect the change. We do not have to restart the service.

So from now on, our CRON service will delete the unnecessary files from the /tmp directory within a specific time frame repeatedly.

Check out blog posts from one of our developers on Linux Shell (bash) Programming Guidelines and 7 Linux Shell (bash) Programming Unusual Syntax that Can Bother.

References:

• https://manpages.ubuntu.com/manpages/lunar/en/man8/tmpreaper.8.html
• https://www.red5pro.com/docs/special/cloudstorage-plugin/file-cleanup/

Experience the iXora Solution difference as your trusted offshore software development partner. We’re here to empower your vision with dedicated, extended, and remote software development teams. Our agile processes and tailored software development services optimize your projects, ensuring efficiency and success. At iXora Solution, we thrive in a dynamic team culture and experience innovation in the field of custom-made software development.

Have specific project requirements? Personalized or customized software solutions! You can contact iXora Solution expert teams for any consultation or coordination from here. We are committed to maximizing your business growth with our expertise as a custom software development and offshore solutions provider. Let’s make your goals a reality.
Thanks for your patience!