Posts Tagged ‘apache’

Manually delete data from Splunk

Thursday, December 27th, 2012

By default Splunk doesn’t delete the logging data that it’s gathered. Without taking some action to remove data it will continue to process all results from Day 1 if doing an All time search. That may be desirable in some cases where preserving and searching the data for historical purposes is necessary, but when using Splunk only as a monitoring tool the older data becomes superfluous after time.

Manually deleting information from Splunk is irreversible and doesn’t necessarily free disk space. Splunk users should only delete when they’ve verified their search results return the information they expect.

Enabling “can_delete”

No user can delete data until he’s been provided the can_delete role. Not even the admin account in the free version of Splunk has this capability enabled. To enable can_delete:

  1. Click the Manager link and then click the Access Controls link in the Users and authentication section.
  2. Click the Users link. If running the free version of Splunk then the admin account is the only account available. Click the admin link. Otherwise, consider creating a special account just for sensitive procedures, such as deleting data, and assigning the can_delete role only to that user.
  3. For the admin account move the can_delete role from the Available roles to the Selected roles section.
    Enable can_delete
  4. Click the Save button to keep the changes.

Finding data

Before deleting data be sure that a search returns the exact data to be deleted. This is as simple as performing a regular search using the Time range drop down menu to the right of the Search field.

The Time range  menu offers numerous choices for limiting results by time including Week to dateYear to date and Yesterday. In this case, let’s search for data from the Previous month:

Search time range

Wait for the search to complete then verify the results returned at the results that need deleting.

Deleting data

Deleting the found data is as simple as performing a search and then piping it into a delete command:

Search and delete

 

This runs the search again, deleting found results on the fly, which is why searching first before deleting is important. Keep in mind that a “delete and search” routine takes as long or longer to run as the initial search and consumes processing power on the Splunk server. If deleting numerous records proceed with one search/delete at a time to avoid overtaxing the server.

Suppressing the PHP Version

Thursday, April 28th, 2011

Yesterday, we looked at hiding the version of Apache being run on a web server. Today we’re going to look at suppressing the version of PHP.

By default, the PHP configuration file, php.ini, is stored at /etc/php5/apache2/php.ini (in most distributions of Linux) or just in /etc/php.ini (as with Mac OS X). In this file

vi /etc/php.ini

Then locate the expose_php variable within the file. Once found, set it to Off as follows:

expose_php = Off

Doing so will not improve the overall security of a system (unless you believe in security through obscurity). However, it is a good idea and will help defeat a number of vulnerability scanners. If you do suppress the Apache and PHP versioning information for the sake of passing a vulnerability scanner on a backported distribution of one of the packages then it would be a good idea to check the CVEs for the port you are using and verify that you are secure.

Hiding the Apache Software Version

Wednesday, April 27th, 2011

By default, Apache displays version information when queried. One aspect of securing Apache servers is to suppress this information from being shown to clients. This also helps immensely with vulnerability scanners that only look at the http header, as many vendors now backport or fork the code for Apache (e.g. Red Hat and Apple).

To do so, one need only make a small change to the httpd.conf file. By default, Apache stores its configuration files in Linux in the /etc/httpd/conf/httpd.conf file. In Mac OS X they can be found at /private/etc/apache2/httpd.conf Here, you will find the ServerTokens and ServerSignature directives. These should be set to ProductOnly and Off respectively, as follows:

ServerTokens ProductOnly
ServerSignature Off

Once these have been changed, you will need to restart the httpd service. One way to do so is to use init.d:

/etc/init.d/httpd restart

To verify that the version number has been suppressed, use telnet:

telnet www.318.com http

Open Source Code Development

Monday, June 5th, 2006

Developers of code have always been fairly open with their tips and tricks. New advancements in the websphere come fast and many of them come from the open source community. Led by people like Linus Torvalds, the original author of Linux, the open source ommunity has rewritten many of the most popular proprietary applications on the market and made them freely available to the world, asking only that if they don’t sell the code you don’t turn around and sell the code as well.

This was the foundation for the web. Apache, the most popular web server in use, is a product of the open source community. Recently, due to a large pool of code to draw upon and the entry into the open source community of many proprietary products we have been seeing a lot of advancements coming at a more rapid rate than ever. OpenOffice.org, a project for replacing Microsoft Office, Eclipse, a project supposedly named because they were going to “eclipse” Sun and a list almost as long as the postings on SourceForge.net (a popular site for open source software) have emerged.

This is changing the way people write code. Programmers today are often charged with assembling and integrating code more than they are actually writing new code. Many organizations have seen that by using code repositories online and in some cases searchable is more efficient than writing new code. In many cases, software developers and architects spend more time finding, downloading and evaluating available code than anything else.

Some programmers sell their code, but many just post it online giving back to the community that helped them find code they have been using and in some cases learn their craft. Finding the appropriate code for a given task and making sure that the licensing and documentation is taken care of can be a tough task. This is where a new type of search engine comes into play. Koders.com currently offers over 225,000,000 lines of code for languages including PHP, Python, SQL and many others. Krugle is another search engine that offers much more information on code although it is currently in beta. If you would rather pay for your ability to search code you can sign up for the protexIP/OnDemand service with Black Duck. Anyone who will be writing a lot of code should get to know all their options for trolling around for code.

Installing AWStats on Mac OS X Server

Friday, February 3rd, 2006

Here are the steps for setting up AWStats on Mac OS X 10.4 Tiger Server.

1. Download the last stable release of AWStats from www.awstats.org to your desktop.
2. In the Finder, navigate to /var/log/httpd
3. Backup and remove any old web logs.
4. Open Server Admin.
5. Select Web:Settings:Modules
6. Make sure the “perl_module” and “php4_module” are enabled.
7. Click Save.
8. Select the “Sites” pane.
9. Double-click the entry for the site you are going to enable stats on.
10. Select the “Options” pane.
11. Enable CGI Execution and Server Side Includes (SSI).
12. Click Save.
13. Select the “Realms” pane.
14. Create a new Realm called “awstats_data” in the site’s root directory or “Web Folder”. If necessary, within the Finder, navigate to the /Library/WebServer/Documents directory and create a new folder called “awstats_data”. (i.e. /Library/WebServer/Documents/awstats_data).
15. Enable Browse/Author access for the local Administrator and the “www” user only.
16. Click Save.
17. Select the “Logging” pane.
18. Change the access logging Format to “combined”
19. Change the access log Location to /var/log/httpd/awstats_access_log
20. Change the error log Location to /var/log/httpd/awstats_error_log
21. Click Save.
22. Select the “Aliases” pane and add 127.0.0.1 as an alias.
23. Click Save.
24. Click the left-arrow icon to exit Editing the site.
25. Make sure the site is enabled and Web Services are running.
26. Open Workgroup Manager.
27. Verify ACLs are enabled on the volume containing the “awstats_data” directory you created earlier.
28. Change the posix permissions of the “awstats_data” directory to allow Read/Write access for the admin group.
29. Create an ACL to allow Read/Write access for the “www” user.
30. Click Save.
31. Close Server Admin and Workgroup Manager.
32. Expand the awstats.zip downloaded from awstats.org to your desktop.
33. Create a new folder named “awstats” in the /Library/WebServer directory.
34. Copy the contents of ~/Desktop/awstats-6.5/ to /Library/WebServer/awstats
35. Open a Terminal session.
36. Type cd /Library/WebServer/awstats/tools
37. Press Return
38. Type sudo perl awstats_config.pl
39. Follow the prompts…

—– AWStats awstats_configure 1.0 (build 1.6) (c) Laurent Destailleur —–
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to ‘understand’ all possible ways to use AWStats…
Read the AWStats documentation (docs/index.html).

—–> Running OS detected: Mac OS

—–> Check for web server install
Found Web server Apache config file ‘/etc/httpd/httpd.conf’

—–> Check and complete web server config file ‘/etc/httpd/httpd.conf’
AWStats directives already present.

—–> Update model config file ‘/Library/WebServer/awstats/wwwroot/cgi-bin/awstats.model.conf’
File awstats.model.conf updated.

—–> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
40. file (required if first install) [y/N] ? y

—–> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
41. site.domain.com

—–> Create config file ‘/Library/WebServer/awstats/wwwroot/cgi-bin/awstats.site.domain.com.conf’
Config file /Library/WebServer/awstats/wwwroot/cgi-bin/awstats.site.domain.com.conf created.

—–> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/Library/WebServer/CGI-Executables/awstats.pl -update -config=site.domain.com
Or if you have several config files and prefer having only one command:
/Library/WebServer/Documents/tools/awstats_updateall.pl now
42. Press ENTER to continue…

A SIMPLE config file has been created: /Library/WebServer/awstats/wwwroot/cgi-bin/awstats.site.domain.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for site.domain.com’ with command:
> sudo perl awstats.pl -update -config=site.domain.com
You will also read your statistics for ‘site.domain.com’ with URL:
> http://localhost/cgi-bin/awstats.pl?config=site.domain.com

43. Press ENTER to finish…
44. Edit the awstats.site.domain.com.conf file (in your favorite text editor, as root) and add these lines or augment existing lines for these variables.
LogFile=”/var/log/httpd/awstats_access_log”
LogType=W
LogFormat=1
SiteDomain=”site.domain.com”
DirData=”/Library/WebServer/Documents/awstats_data”
DirCgi=”/Library/WebServer/CGI-Executables”
DirIcons=”/icon”
AllowToUpdateStatsFromBrowser=1
AllowFullYearView=3
46. Move the remaining contents of /Library/WebServer/awstats/wwwroot to /Library/WebServer/Documents
47. Move the “tools” directory of /Library/WebServer/awstats to /Library/WebServer/Documents
48. Open Terminal
49. Type cd /Library/Webserver/CGI-Executables/
50. Type sudo perl awstats.pl -update -config=site.domain.com
51. From the server, open a browser and go to the site http://localhost/cgi-bin/awstats.pl?config=site.domain.com
52. If you see the data then you know that both your configuration and log file format is good.
53. Now it’s time to tell the system to update awstats on a regular basis.
Create a CRON job to run the command /Library/WebServer/CGI-Executables/awstats.pl -update -config=site.domain.com