Digital Democracy Redesign Mockups

Digital Democracy is like YouTube for local government hearings, bolstered with a splash of artificial intelligence. Bots create transcripts of lawmakers’ every official utterance at the state house and use face recognition software to keep track of who’s speaking. Voters can search the transcripts by speaker and subject while at the same time getting a glimpse of legislators’ financial ties.

Wired Magazine Feb 2017

The Digital Democracy site allows clear and comprehensible access to important data about elected State officials: records of how they voted, who contributed money to their campaigns, relationships with lobbyists, video records of hearings and testimonies, and transcriptions of the same. CivicActions lead the effort to rebuild the existing Drupal 7 platform in Drupal 8, and transform the single-state architecture to a scalable, real-time, multi-state, re-brandable solution.

Why Drupal was chosen: 

Given the existing Drupal platform, the motivations behind choosing Drupal 8 instead of building further upon 7 were:

  • Custom Entities: The Drupal 7 site relied upon a lengthy nightly custom import from their main data warehouse, with Drupal 8’s support for custom entities, that was transformed to a direct database connection - providing real-time data.
  • OOP Architecture: The structure of entities called for many slight variations between each state’s legislative setup. Drupal 8’s ability to extend and override classes reduced code duplication and simplifies future customizations.
  • Configuration Management: Utilizing many of Drupal 8’s CMI features allowed for simplified deployment and local development.
Describe the project (goals, requirements and outcome): 

PROJECT NEEDS:

  • REAL-TIME DATA/MULTI-STATE: The previous platform depended on a lengthy nightly import process which hampered the site’s ability to deliver hearing transcriptions right as they became available. The tool was also specific for California - the user interface and Drupal 7 codebase were not designed to be scaled.
  • SITE SEARCH: The site called for a centralized search, but of many different types of data. The existing search capabilities had an inconsistent presentation and required the user to navigate to different areas before starting a search.
  • RESPONSIVE DESIGN: The existing site was not designed responsively and was not very usable on mobile devices, excluding a large pool of potential visitors. The new site called for a responsive, accessible design that could also allow for customizations for multi-state use.
  • BRANDABLE: Another important client goal was to create an extensible platform that could be rebranded as a “white label” solution for advocacy groups and organizations.
  • HIGHEST DEVOPS STANDARD: The existing platform had a lengthy, manual deployment process that made QA and local development difficult. In order to support future growth, the platform needed to have a solid DevOps process for automated deployment and testing.

RESULTS:

A powerful, scalable platform for the public to engage with their state’s legislation without having to attend hearings.

  • EXTERNAL DATABASE INTEGRATION: Digital Democracy’s team of developers and data analysts maintain all of the video, hearings, transcriptions, and analysis data in their own database schema - spanning over 100 tables and over 20+ million rows. Through a comprehensive technical architecture redesign, seamless usage of their data by Drupal 8 custom entities treats the data as if it were local.
  • EASE OF DEVELOPMENT: By bringing the external data into Drupal structures, the development of new data presentations, analysis, and user experience were simplified, allowing their staff to focus on development rather than maintaining an import process that was prone to issues.
  • DEVOPS/DEPLOYMENT: By utilizing Jenkins, Drupal 8 Configuration Management, and GitHub, deployment for QA/Dev/Production environments was reduced to 3 minutes - all triggerable with a single click. Docker and DrupalVM environments built into the codebase allowed for local development on any platform.
  • USER INTERFACE: A new responsive theme based on Zen was created, utilizing KSS node style guides, component library design, and template override capabilities aware of multi-state and site type.
  • SCALABILITY: Transformation to a Drupal 8 platform allowed for real-time data, eliminating the import process, and extreme extensibility. A single codebase was developed that provided the means to support any number of states, including separate overrides for visual elements. These changes created a stable, maintainable codebase for growth.
  • SEARCH: Utilizing Apache Solr for faceted search, compound filtering, and advanced text search capabilities created a much improved user experience. A powerful site-wide search autocomplete system was created using a single search box to simultaneously give results for Bills, Hearings, Speakers, Committees, and Organizations - quickly directing users to the content that they are seeking.
  • WHITELABEL/ACTION CENTER: The same platform was made to double as a deployable solution where a site manager has full control of what content a visitor will see, plus an “Action Center” where campaigns can be created and focused on a particular bill. Video clips of hearings, downloads, and other shareables set up by administrators could be used by the audience to bolster the campaign/message.
Modules/Themes/Distributions
Why these modules/theme/distribution were chosen: 

Much of the heavy lifting was done using built in core modules plus custom modules. Paragraphs allowed for flexibility in creating dynamic sets of field data, while views_field_view handled some very complex view combinations of year-grouped data.

The nature of hearing data coming from each state leaned towards providing a user with the tools to find relevant information easily. Through the use of Apache Solr and Search API combined with Facets, the UI was able to provide easy to use, filterable search results very quickly.

The Zen theme was chosen as the base for a responsive design that never existed in the D7 version, including development of a KSS living styleguide.

Organizations involved: 
Community contributions: 

AdvAgg - S3FS Incompatibility: https://www.drupal.org/node/2842163
Config Ignore - Block Regions Ignore: https://www.drupal.org/node/2859452
Config Partial Export: https://www.drupal.org/project/config_partial_export
Config Removal: https://www.drupal.org/sandbox/nerdstein/2836465
Facets - Dropdowns: https://www.drupal.org/node/2815269
Facets - Chosen: https://www.drupal.org/node/2815271
Glossary - Facets Compatibility: https://www.drupal.org/node/2796649
HTML Field Formatter: https://www.drupal.org/sandbox/nerdstein/2826175
Multiselect - Module into 1.x-dev: https://www.drupal.org/project/multiselect
Request Parameter Condition: https://www.drupal.org/sandbox/nerdstein/2834882
Search API - Computed Fields: https://www.drupal.org/node/2814925
Views Field View - Override view/display: https://www.drupal.org/node/2819945
Views Field View - Field Token caps: https://www.drupal.org/node/2793785
Views Field View - Non-token args: https://www.drupal.org/node/2824359
Views Summary Rows - https://www.drupal.org/sandbox/nerdstein/2827911
BADCamp 2016 Presentation “BFFs 4eva: Drupal 8 and Custom Databases”- https://2016.badcamp.net/session/bffs-4eva-drupal-8-and-custom-databases

Project team: 

CivicActions

Project Manager: Steve Curtis
Tech Lead/Architect: Kristian Ducharme (Acquia D8 Back End Specialist Certified)
UI/Design/Development: Iris Ibekwe (Acquia D8 Front End Specialist Certified), Alex Scott, Kev Walsh
DevOps Support: David Numan
Developers: Owen Barton, Adam Bergstein (Acquia D7 Grand Master Certified), Andrew Hawks (Acquia D7 Developer/Back End Specialist Certified), Josh Rose, Steve Wirt

IATPP (Institute for Advanced Technology and Public Policy)

Sam Blakeslee
Christine Roberts
Dr. Foaad Khosmood
Dr. Alex Dekhtyar
Dr. Davide Falessi
Toshihiro Kuboi

View of Hearings Search
Hearing Video Display
Speaker Profile Page