Data Analytics And Technical Expert Services (DANTES)
- URL: dantesportal.org
- Started: October 2012
- Deployed: July 2014
- Role: Team leader, full-stack developer
- Team Size: 4
- Tech: Drupal 6.x/7.x, PHP, Apache Solr, Amazon AWS (EC2), LaTeX
Note: This write-up mostly concerns the initial construction and launch of DANTES. Other work on this project is covered in separate essays.
This originally started as a warm-up project, when I first got to the USPS OIG; it was an old idea that a previous developer had started on but never finished. Initially, the plan was to just finish what the previous developer started, and leave it at that. However, once that was done, this turned into an ongoing side project, with continuous work throughout my entire tenure at the USPS OIG.
The original concept for this site was intriguing: Create a platform for the Offices of Inspectors General to share resources and knowledge, to make all of us more effective at our mission. For those unaware, Offices of Inspectors General are federal agencies responsible for auditing and policing the workforce of specific other federal agencies, to investigate fraud, waste, and abuse. Each federal agency has an Office of Inspector General (OIG), meaning that there's over a hundred OIGs. Additionally, there are agencies like the Council of Inspectors General for Integrity and Efficiency (CIGIE), which are part of the OIG community but not really attached to a specific agency. And since all the OIGs are independent from the agency they oversee, they vary greatly in size and budget; some employ thousands of people in nationwide field offices and have highly advanced capabilities, some are as small as 1-3 people and basically have a one-room office in their jurisdiction agency's headquarters. So, even though all OIGs share the same mission (just with different jurisdictions), there are things that some OIGs can do in-house (forensics, surveillance, Big Data analysis, GIS, etc), but others cannot.
Thus, the mission of DANTES was to level the playing field. We wanted to build a shared website, accessible to all personnel at all OIGs, where OIGs with advanced capabilities could share those capabilities with smaller OIGs, which is a win-win for all concerned; a large OIG with downtime gets to keep their people busy, and a small OIG that needs extra help can work with a fellow OIG instead of having to run everything through the FBI. This mission evolved over time to also include facilitating inter-agency collaboration and information sharing, via community features like forums, blogs, and a document library.
- Build a shared website open to all Offices of Inspectors General.
- Incorporate a system for OIGs to request law enforcement and audit/analytical services from each other.
- Build forums, blogs, and a document library that all users could contribute to.
- Build a community calendar for inter-agency events, like training sessions and collaborative summits.
My initial work on this project started by analyzing what had already been done, which gave me ideas, but most of it was not ready for production use. The main focus was the service sharing/request system, which I initially conceptualized as a helpdesk-style support ticketing system, and that's exactly how I built it. The rest was fairly straightforward, using off-the-shelf Drupal modules; I stuck with the initial choice of Drupal 6.x, because some of the modules I needed for the service request system had not yet been ported to Drupal 7.x.
I also had to write a standardized version of the Memorandum of Understanding (MOU) drafted by our legal team for similar purposes; it's a document that allows federal agencies to collaborate, and every agency that requested services from another had to have one with that agency, so we needed an example form that could easily have any agency's information added to it. To make this process easier, and potentially make it something that the DANTES website could programmatically generate, I wrote it as a LaTeX document, so that simply changing variables would adjust the document as-needed, and the result could be compiled into a nicely-styled PDF file.
After some extensive security testing, the site was pretty much ready for launch within six months. Around that same time, my primary projects started to really ramp up, so my work on DANTES was sidelined, pending a green light for launch. Roughly six months later, we received that green light, but with a requirement that the site be rebuilt using Drupal 7.x, and my role shifted from developer to team leader; I didn't do much hands-on development work during that portion, but I was responsible for coordinating the efforts of two developers, one content curator/subject-matter expert, and an information security analyst. During the initial launch, the service request system was set aside due to the aforementioned module that had not been ported to Drupal 7.x, but it was later re-added, and the site gained features overall, with new content areas to facilitate more types of inter-agency collaboration. We set up hosting for the site using Amazon EC2, and deployed in mid-July 2014.
Following deployment, I continued to be part of the team responsible for this site, and I was in a team leader role for new features. For example, shortly after deployment, I oversaw efforts to build a search indexing system for the site using Apache Solr, with Apache Tika for parsing document files. However, I did resume active hands-on development on it in November 2015, during a two-month project to redesign the appearance of the site; I was primarily in charge of the back-end portion of that project, customizing template files and views, with assistance from another developer who specializes in front-end/layout work. Unfortunately, as of Summer 2016, that work has not yet been deployed to the production version of the site, but the results were greatly improved over the site's current design, and much more user-friendly.