Event Report: FudConIn 2015

As promised I am back from fudcon india with loads of experience and new knowhow about various tools.

::DAY ONE::

The first day started with keynote talk by Dennis Gilmore on “Delivering Fedora for everything and everyone”. He discussed about the future plans for Fedora release engineering team. Next I moved to a different room where a talk on GlusterFS was being held by Vikhyat Umrao. This was the first time I heard about Gluster File System so was pretty much curious  to learn more about it. After that I attended back to back two more talks on GlusterFS, which are  “Geo-Replication and Disaster Recovery in GlusterFS” by Bipin Kunal and “Efficient data maintenance in GlusterFS using Databases” by Joseph Elwin Fernandes . I must add session by Joseph Elwin Fernandes was great, he is a really good speaker 🙂 . After lunch i attended Jared Smith’s talk on “Whats new in Drupal 8?”. I learned about new features that are going to be added to Drupal 8, like REST support, editing content directly, easy installation and language selection etc. The day ended with a keynote talk by Harish Pillay on how to evaluate open source projects and spoke about “open source prospector” a tool to track FOSS projects over the globe.

::DAY TWO::

Day two started with a keynote by Jiri Eishchmann on the future of fedora workstation. He spoke about better graphics support, better battery life in F23 also more reliable weekly updates. Then there was a session on Haskell by Jens Petersen. Next I attended Rejy Cyriac’s talk on Selinux. after which i too configured my selinux for the good 🙂 . Soon after that a interesting workshop was done by Mayur Patil on how compile linux kernel and wrote a small test kernel module. Day two ended by tenzin chokden with his keynote talk on how GhostNet affected the Tibetian Community and how Linux helped them stop GhostNet from spreading.

::DAY THREE::

Day three was mostly meeting people and attending fewer talks and workshops. Aditya Patawari, Lalatendu Mohanty took a workshop on Docker basics. Next Flask 101 workshop was taken by Sayan Chowdhury and Ratnadeep Debnath. Finally closing ceremony by Rupali Talwatkar after the lunch. Oh yes! also a photo session was there 😀 .

Will update with the link to the talks soon.

::ABOUT PUNE::

Pune is a great place! Had awesome food! Enjoyed alot! Also was almost lost on my way to the venue since there were two MITs and I reached the wrong one! Light drizzle on the day of return. Awesome experience altogether! Waiting for next Fudcon India 🙂 .

PS: Pics coming soon

Event Report: FudConIn 2015

Finally integrating Gcov and Lcov tool into Cppagent build process

This is most probably my final task on Implementing Code Coverage Analysis for Mtconnect Cppagent. In my last post i showed you the how the executable files are generated using Makefiles. In Cppagent the Makefiles are actually autogenerated by a cross-platform Makefile generator tool CMakeTo integrate Gcov and Lcov into the build system we actually need to start from the very beginning of the process which is cmake. The CMake commands are written in CmakeLists.txt files. A minimal cmake file could look something like this. Here we have the test_srcs as the source file and agent_test as the executable.


cmake_minimum_required (VERSION 2.6)

project(test)

set(test_srcs menu.cpp)

add_executable(agent_test ${test_srcs})

Now lets expand and understand the CMakeLists.txt for cppagent.

set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../agent/CMake;${CMAKE_MODULE_PATH}") 

This sets the path where cmake should look for files when files or include_directories command is used. The set command is used to set values to the variables. You can print all the available variable out using the following code.

get_cmake_property(_variableNames VARIABLES)
foreach (_variableName ${_variableNames})
    message(STATUS "${_variableName}=${${_variableName}}")
endforeach()

source: stackoverflow.com

Next section of the file:

if(WIN32)
 set(LibXML2_INCLUDE_DIRS ../win32/libxml2-2.9/include )
 
 if(CMAKE_CL_64)
 set(bits 64)
 else(CMAKE_CL_64)
 set(bits 32)
 endif(CMAKE_CL_64)
 
 file(GLOB LibXML2_LIBRARIES "../win32/libxml2-2.9/lib/libxml2_a_v120_${bits}.lib")
 file(GLOB LibXML2_DEBUG_LIBRARIES ../win32/libxml2-2.9/lib/libxml2d_a_v120_${bits}.lib)
 set(CPPUNIT_INCLUDE_DIR ../win32/cppunit-1.12.1/include)
 file(GLOB CPPUNIT_LIBRARY ../win32/cppunit-1.12.1/lib/cppunitd_v120_a.lib)
endif(WIN32)

Here, we are checking the platform we are working on and accordingly the library variables are being set to the windows based libraries. We will discuss the file command later.

if(UNIX)
 execute_process(COMMAND uname OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE CMAKE_SYSTEM_NAME)
 if(CMAKE_SYSTEM_NAME MATCHES Linux)
 set(LINUX_LIBRARIES pthread)
 endif(CMAKE_SYSTEM_NAME MATCHES Linux)
endif(UNIX)

Next if the OS platform is Unix based then we execute the command uname as child-process and store the output in CMAKE_SYSTEM_NAME variable. If its a Linux environment., Linux  will be stored in the CMAKE_SYSTEM_NAME variable, hence,  we set the variable LINUX_LIBRARIES to pthread(which is the threading library for linux). Now we find something similar we did in our test CMakeLists.txt. The project command sets the project name, version etc. The next line stores the source file paths to a variable test_src

set( test_srcs file1 file2 ...)
Now we will discuss about the next few lines.
file(GLOB test_headers *.hpp ../agent/*.hpp)

The file command is used to manipulate the files. You can read, write, append files, also GLOB allows globbing of files which is used to generate a list of files matching the expression you give. So here wildcard expression is used to generate a list of all header files in the particular folder *.hpp.

include_directories(../lib ../agent .)

This command basically tells cmake to add the directories specified by it to its list of directories when looking for a file.

find_package(CppUnit REQUIRED)

This command looks for package and loads the settings from it. REQUIRED makes sure the External package is loaded properly else it must stop throwing an error.

add_definitions(-DDLIB_NO_GUI_SUPPORT ${LibXML2_DEFINITIONS})

add_definitions is where the additional compile time flags are added.

add_executable(agent_test ${test_srcs} ${test_headers})

This line generates an executable target for the project named agent_test and test_src and test_headers are its source and header files respectively. 

target_link_libraries(agent_test ${LibXML2_LIBRARIES} ${CPPUNIT_LIBRARY} ${LINUX_LIBRARIES})

This line links the executable its libraries.

::Gcov & Lcov Integration::

Now that we know our CMake file well, lets make the necessary changes.

Step #1

Add two variables and set the appropriate compile and linking flags for gcov and lcov respectively.

set(GCOV_COMPILE_FLAGS "-fprofile-arcs -ftest-coverage")
set(GCOV_LINK_FLAGS "-lgcov")

Step #2

Split the source into two halves one being the unit test source files and the other being the cppagent source files. We are not interested in unit test files’ code coverage.

set( test_srcs test.cpp
 adapter_test.cpp
 agent_test.cpp
 checkpoint_test.cpp
 config_test.cpp
 component_test.cpp
 component_event_test.cpp
 connector_test.cpp
 data_item_test.cpp
 device_test.cpp
 globals_test.cpp
 xml_parser_test.cpp
 test_globals.cpp
 xml_printer_test.cpp
 asset_test.cpp
 change_observer_test.cpp
 cutting_tool_test.cpp
 )
set(agent_srcs ../agent/adapter.cpp 
 ../agent/agent.cpp 
 ../agent/checkpoint.cpp
 ../agent/component.cpp 
 ../agent/component_event.cpp 
 ../agent/change_observer.cpp
 ../agent/connector.cpp
 ../agent/cutting_tool.cpp
 ../agent/data_item.cpp 
 ../agent/device.cpp 
 ../agent/globals.cpp 
 ../agent/options.cpp
 ../agent/xml_parser.cpp 
 ../agent/xml_printer.cpp
 ../agent/config.cpp
 ../agent/service.cpp
 ../agent/ref_counted.cpp
 ../agent/asset.cpp
 ../agent/version.cpp
 ../agent/rolling_file_logger.cpp
 )

Step #3

Like i told in Step 2 we are not interested in unit test source files. So here we just add the Gcov compile flags to only the cppagent source files. So .gcno files of only the agent source files are generated.

set_property(SOURCE ${agent_srcs} APPEND PROPERTY COMPILE_FLAGS ${GCOV_COMPILE_FLAGS})

Step #4

Now we also know that for coverage analysis we need to link the “lgcov” library. Therefore, we do this in the following way.

target_link_libraries(agent_test ${LibXML2_LIBRARIES} ${CPPUNIT_LIBRARY} ${LINUX_LIBRARIES} ${GCOV_LINK_FLAGS}) 

Step #5

Since we love things to be automated. I added a target for the make command to automate the whole process of running test and copying the “.gcno” files and moving the “.gcda” files to a folder then running the lcov command to read the files and prepare a easily readable statistics and finally the genhtml command to generate the html output. add_custom_target allows you to add custom target for make(Here i added “cov” as the target name). COMMAND allows you to specify simple bash commands.

add_custom_target( cov
COMMAND [ -d Coverage ]&&rm -rf Coverage/||echo "No folder"
COMMAND mkdir Coverage
COMMAND agent_test
COMMAND cp CMakeFiles/agent_test.dir/__/agent/*.gcno Coverage/
COMMAND mv CMakeFiles/agent_test.dir/__/agent/*.gcda Coverage/
COMMAND cd Coverage&&lcov -t "result" -o cppagent_coverage.info -c -d .
COMMAND cd Coverage&&genhtml -o coverage cppagent_coverage.info
COMMENT "Generated Coverage Report Successfully!"
)

::Conclusion::

Now to build test and generate report.

Step #1 cmake .    // In project root which cppagent/
Step #2 cd test    // since we want to build only test
Step #3 make       // This will build the agent_test executable.
Step #4 make cov   // Runs test, Copies all files to Coverage folder, generates report.

So, we just need to open the Coverage/coverage/index.html to view the analysis report. Final file will look something like this.

Finally integrating Gcov and Lcov tool into Cppagent build process

Using Gcov and Lcov to generate Test Coverage Stats for Cppagent

In my last post we generated Code coverage statistics for a sample c++. In this post i will be using gcov & lcov to generate similar code coverage for tests in cppagent. To use gcov we first need to compile the source files with --coverage flag. Our sample c++ program was a single file so it was easy to compile, but for cppagent they use makefiles to build the project. Hence, i started with the Makefile looking for the build instructions.

If my previous posts i discussed the steps for building the agent_test executable, which starts by running make command in test folder. So i started tracing the build steps from the Makefile in test folder. Since we run make without any parameters, the default target is going to be executed.

The first few lines of the file were as below.

# Default target executed when no arguments are given to make.

default_target: all

.PHONY : default_target

These lines specifies that the default_target for this build is all. On moving down the file we see the rules for all.

# The main all target

all: cmake_check_build_system

cd /home/subho/work/github/cppagent_new/cppagent && $(CMAKE_COMMAND) -E cmake_progress_start /home/subho/work/github/cppagent_new/cppagent/CMakeFiles /home/subho/work/github/cppagent_new/cppagent/test/CMakeFiles/progress.marks

cd /home/subho/work/github/cppagent_new/cppagent && $(MAKE) -f CMakeFiles/Makefile2 test/all

$(CMAKE_COMMAND) -E cmake_progress_start /home/subho/work/github/cppagent_new/cppagent/CMakeFiles 0

.PHONY : all

So here in the line

cd /home/subho/work/github/cppagent_new/cppagent && $(MAKE) -f CMakeFiles/Makefile2 test/all

We can see Makefile2 is invoked with target test/all.

In Makefile2 towards the end of the file we can see the test/all target build instructions as,

# Directory level rules for directory test

# Convenience name for "all" pass in the directory.

test/all: test/CMakeFiles/agent_test.dir/all

.PHONY : test/all

The rule says to run the commands defined under target test/CMakeFiles/agent_test.dir/all. These commands are:

test/CMakeFiles/agent_test.dir/all:

$(MAKE) -f test/CMakeFiles/agent_test.dir/build.make test/CMakeFiles/agent_test.dir/depend

$(MAKE) -f test/CMakeFiles/agent_test.dir/build.make test/CMakeFiles/agent_test.dir/build

$(CMAKE_COMMAND) -E cmake_progress_report /home/subho/work/github/cppagent_new/cppagent/CMakeFiles 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

@echo "Built target agent_test"

.PHONY : test/CMakeFiles/agent_test.dir/all

The first two lines run the build.make file with target ‘test/CMakeFiles/agent_test.dir/depend‘ and ‘test/CMakeFiles/agent_test.dir/build‘ . The build.make contains all the compile instructions for each of the c++ files. This file is in ‘test/CMakeFiles/agent_test.dir’ folder along with flag.make , link.txt etc files. The  flag.make file contains all the compile flags and the ‘link.txt‘ contains the libraries flag needed by linker. On adding the --coverage flag to these files we can make the c++ source files compile with gcov linked hence .gcno files are generated when the make command is run.

After that we need to run the agent_test as usual. This will create the data files .gcda files. After that we need to gather the .gcda and .gcno files together and run the lcov and genhtml commands and then the html output will be obtained.

This slideshow requires JavaScript.

Using Gcov and Lcov to generate Test Coverage Stats for Cppagent

10 years of DGPLUG

We recently celebrated the 10 glorious years of DGPLUG. On this occasion we had a 5 day workshop (29th August to 2nd September) at NIT Durgapur.

::Day 1::

The first day was more or less introductory session where we discussed about what are our goals , our history and our programme for the future. The attendees constituted mainly of 1st,2nd and 3rd year engineering students of various colleges. The speaker for the session was Kushal Das, he told them about the Summer Training conducted by DPLUG for free every year. We told them about Open Source and how they can contribute to opensource projects. The next talk was by Praveen Kumar and was all about Fedora Project and how we can contribute to it. Next we had a really interesting talk by P.J.Prasad on Iptables. At the end we concluded the first day of the workshop with light discussions on various topics and also asked the students to get some packages installed for the next day.

::Day 2::

On the day two of the event the auditorium was teeming with eager faces , a workshop on Python was to be held. Kushal Das took the session on python along with the introduction of Vim. We had a tough time helping them to this new editor nevertheless we enjoyed doing it. Python for you and me book was followed in the session. We covered basic python commands , data structures and other basic functions of the language. By the end of the day we managed to write a program which could list down the files and folders of a directory quite similar to ls command of linux.

::Day 3::

On the day three we had workshops on flask, a web-framework based on python by Sayan Chowdhury in the first half of the day. Later In the second half we had workshop on unit test module of python by Ratnadeep Debnath where we learned to write test cases for our functions.

::Day 4::

On the Fourth day of the event we had a session on documenting our codes by Kushal Das. We introduced reStructuredText to write our documentation and then use rst2s5 to convert them to presentations. After that we used a powerful python package sphinx and prepared a demo documentation.

::Day 5::

The day five was all about how to contribute to upstream projects, Learn using Git, making patches, and other important git features and commands. After that we had a general discussion session followed by feedback session. We got many positive feedbacks and suggestions for improvement which has been noted down and will be kept in mind in the upcoming events.

Finally we talked about various projects in hand, and newer project ideas to be developed. Hoping to meet these awesome people soon at Pycon India. Till then Keep Coding! 😉

10 years of DGPLUG

FOSS (Free & Open Source Software)

Free and open-source software (F/OSS, FOSS) or free/libre/open-source software (FLOSS) is software that is both free software and open source. It is liberally licensed to grant users the right to use, copy, study, change, and improve its design through the availability of its source code. This approach has gained both momentum and acceptance as the potential benefits have been increasingly recognized by both individuals and corporations.

Free software, software libre or libre software is software that can be used, studied, and modified without restriction, and which can be copied and redistributed in modified or unmodified form either without restriction, or with restrictions that only ensure that further recipients have the same rights under which it was obtained and that manufacturers of consumer products incorporating free software provide the software as source code. The word free in the term free software refers to freedom (liberty) and is not at all related to monetary cost.

FOSS (Free & Open Source Software)