Using {OpenGrok to boost up source code reading
Reading XNU kernel source(BSD portion) is one of my mainly job when I developing a generic kernel extension(in Apple’s words)
Apple’s source browser mainly generated by GNU enscript for highlighting, yet it doesn’t support cross reference
It’s painful when you want to cross reference a function, say, its definition, usage, etc.
As I investigated through later, I found a guy setup a XNU kernel cross reference site based on OpenGrok
It really unleashed source code reading, which eventually make my life easier :-)
P.S. Apple is ruining the good reputation over BSD documentations, it makes the BSD portion obscured, so you have to read the source code.
Inspired by antekone’s cross-reference search site, I finally decided I should setup one such by my own for source reading and learning.
Out of source reading purpose and maximum simplicity, I decided to use FreeBSD as the backend server.
.: the following configration is based on FreeBSD, other BSD variants follow the similar manner.
If you prepare to use Linux/Solaris, please move your step to OpenGrok wiki.
Once you logged in you FreeBSD server(taking 11.1 as example), you may change your default shell(which is /bin/sh) to bash
# pkg update
# pkg upgrade
# pkg install bash bash-completion
# chsh -s $(which bash)
And change current user(root) shell to bash, and then reloggin your server.
Install basic dependencies
[root@bsd ~]# pkg install ctags tomcat8 zip unzip
Config and run tomcat
Change HTTP server port to 80 in /usr/local/apache-tomcat-8.0/conf/server.xml
--- server.xml.orig 2018-04-12 12:33:40.450709000 +0000
+++ server.xml 2018-04-12 12:33:58.002425000 +0000
@@ -66,7 +66,7 @@
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
-->
- <Connector port="8080" protocol="HTTP/1.1"
+ <Connector port="80" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
# Oneshot start
[root@bsd ~]# service tomcat8 onestart
# Or -- persistent service see: init(8)
[root@bsd ~]# echo 'tomcat8_enable="YES"' >> /etc/rc.conf
[root@bsd ~]# service tomcat8 start
You may open http://127.0.0.1(replace with your server IP) to check if tomcat is running.
Download {OpenGrok
Generally, you should download the latest stable OpenGrok release
[root@bsd ~]# cd /var
[root@bsd /var]# fetch https://github.com/oracle/opengrok/releases/download/1.1-rc22/opengrok-1.1-rc22.tar.gz
[root@bsd /var]# tar xf opengrok-1.1-rc22.tar.gz
[root@bsd /var]# mv opengrok-1.1-rc22 opengrok
[root@bsd /var]# rm opengrok-1.1-rc22.tar.gz
Configurate {OpenGrok
# TODO:
[root@bsd /var/opengrok]# mkdir -p etc src data web/source
Apply the following patch over /var/opengrok/bin/OpenGrok
--- OpenGrok.orig 2018-04-11 14:44:10.165926000 +0000
+++ OpenGrok 2018-04-11 14:46:41.236225000 +0000
@@ -566,6 +566,7 @@
javaHome=`dirname $javaHome`
fi
;;
+ FreeBSD:*) javaHome="/usr/local/openjdk8" ;;
esac
if [ -z "${javaHome}" ]
Extract default web application package
[root@bsd /var/opengrok]# cp lib/source.war web/
[root@bsd /var/opengrok/web]# unzip source.war -d source
Apply the following patch for /var/opengrok/web/source/WEB-INF/web.xml
--- web.xml.orig 2018-04-11 15:13:22.680422000 +0000
+++ web.xml 2018-04-11 15:19:09.897436000 +0000
@@ -12,6 +12,19 @@
<param-name>ConfigAddress</param-name>
<param-value>localhost:2424</param-value>
</context-param>
+
+ <context-param>
+ <description>Source Directory containing all repositories</description>
+ <param-name>SRC_ROOT</param-name>
+ <param-value>/var/opengrok/src</param-value>
+ </context-param>
+
+ <context-param>
+ <description>Data Directory for OpenGrok</description>
+ <param-name>DATA_ROOT</param-name>
+ <param-value>/var/opengrok/data</param-value>
+ </context-param>
+
<listener>
<listener-class>org.opensolaris.opengrok.web.WebappListener</listener-class>
</listener>
Repackage the web application and deploy to tomcat
[root@bsd /var/opengrok/web]# zip -r source.war source/*
[root@bsd /var/opengrok/web]# cp source.war /usr/local/apache-tomcat-8.0/webapps
Source code indexing
As we configurated in web.xml, SRC_ROOT contains all source repositories.
We put all source code into that directory, repository per directory.
Taking libev as an example, I’ll illustrate how to indexing it.
[root@bsd /var/opengrok/src]# fetch http://dist.schmorp.de/libev/libev-4.24.tar.gz
[root@bsd /var/opengrok/src]# tar xf libev-4.24.tar.gz
# The indexing may take minutes or even hours to go -- be patient
[root@bsd /var/opengrok/bin]# ./OpenGrok index
When indexing finished, free free to visit http://127.0.0.1/source/xref(Replace with your server IP)
Tips
-
Indexing multiple versions of a OSS, it usually a good idea to index initial, stable, latest versions.
-
Deploy an annotated version for learning, for example, libev-4.24-comment
-
/usr/local/apache-tomcat-8.0/webapps/ROOT contains the default pages, change to yours
-
As I tested, Firefox is outperform Chrome for viewing large files.
# Change your time zone
[root@bsd ~]# ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# Put the indexing to background if the project is far too large
[root@bsd /var/opengrok/bin]# nohup ./OpenGrok index &
# Setup persistent swap see: https://www.freebsd.org/doc/handbook/adding-swap-space.html
[root@bsd]# dd if=/dev/zero of=/swap0 bs=1m count=512
[root@bsd]# chmod 0600 /swap0
[root@bsd]# echo "md99 none swap sw,file=/swap0,late 0 0" >> /etc/fstab
[root@bsd]# swapon -aL
[root@bsd]# swapinfo # Show swap info
- Instead of extract tarballs, you can use SCM software like git, cvs, svn to clone repositories. Which enable history/annotate features.
Here is a live OpenGrok cross reference research site FYI
Happy groking! :-D
References
Super User’s BSD Cross Reference
antekone’s cross-reference search site (XNU kernel)