When we’re not coding and consulting, our team enjoys getting together in pursuit of fun. Check out what we’re up to outside the office:
SNOW is a Weather Phenomenon & a State of Mind
At the start of the new year, we reintroduced a collaborative three-month wellness initiative: SNOW (Start Now On Wellness). The effort is designed to encourage participation in healthy activities for the body, mind, and soul. We share activities, personal progress reports, and more. There is a monthly prize drawing for participants. The biggest reward? — Inspiration from the team.
#Siplocal — Brewery Tour & Tasting
We enjoyed a taste of Ireland and Baltimore at theGuinness Open Gate Brewery this winter. Our guided tour group learned all about how one of the world’s most loved beers is made here and in Dublin, the history of the Baltimore facility (originally a distillery, it was the first to open in the U.S. after Prohibition ended), and more. There were tastings, slow pours, and delicious Maryland seafood appetizers as well.
Start Your Engines — The Clarity Cup Returns
A handful of years ago, we began what became a racetrack tradition, Clarity’s outing to The Autobahn Indoor Speedway in Jessup. In a friendly and fast competition, we race around the track in electric go-karts. After three rounds, one of us is declared the winner of The Clarity Cup. This year’s race is sure to be another thrilling championship. We’re heading back to the racetrack for our fifth annual Clarity Cup in April.
Great Wine & Food in Annapolis
It’s no secret that we’re into teamwork and business partnerships… We also enjoy a perfect pairing. That’s why every May, when the weather is just right, we plan an outing to Annapolis. Employees and their guests enjoy delicious appetizers and wines that complement the food.
#claritygear Photo Contest
We love our Clarity gear. Thealmost-IG-famous #claritywaterbottle kicked things off a few years ago with appearances in our travel photos and sports pics. To share team fun, every summer, we run a contest — employees post photos of Clarity gear on social media and tag them with #claritygear, #claritywaterbottle, #claritymug, or #clarYETI. Each month, May through September, we draw a winner for a gift card.
We recently contributed to a very special Backpack Buddies program through the local charity, Services from the Heart. Clarity donated $100 per employee, along with an additional amount to round our total gift to the organization up to $3,500.
Healthy Meals for Children in Need
Every week, Services from the Heart provides bags with healthy meals and snacks for children in need. The food packs are given to some of the students who receive free or reduced lunch at specific Anne Arundel County schools to sustain them over weekends, holidays, and during school breaks.
Food items that might be included in a bag are raisins, peanut butter, fresh fruit, breakfast bars, and shelf-stable milk.
About $300 pays for a year of food for one child. Our donation is estimated to cover more than six weeks of supplies for 655 food bags.
To have even more impact, we look forward to volunteering with Services from the Heart later this year.
Local Food Insecurity
According to Services From the Heart, there are more than 16,000 children in Anne Arundel County living with food insecurity. The Backpack Buddies program not only provides nutrition and comfort, it also aims to reduce the impact of poverty by helping meet essential needs so that kids can learn and thrive.
From the Heart
Based in Severna Park, near our HQ, Services from the Heart is supported by many local businesses, Girl Scout and Boy Scout troops, and private citizens. We are proud to join such groups in making a difference. The organization uses the Severna Park Community Center as a packing and storage facility. Please visit the charity’s web site to learn more about their important community work.
Inspired to Give More…
In addition to our Services from the Heart contribution, we have participated in a range of charitable golf outings, supported AFCEA Central Maryland’s scholarship fundraising, and sponsored Severna Park High School sports and its celebrated Rock ‘n’ Roll Revival, as well as a ski race with Seven Springs’ ski racing team — WPRC. TJ served on the selection committee for AFCEA’s Central Maryland STEM Kickstarter grant program.
Clarity also donated to a number of nonprofits through our employee Charitable Matching Program. The organizations that benefitted included:
ICYMI: our team gave a NiFi presentation and demonstration at January’s CodeMash conference. The endeavor, put together by collaborators Ryan Hendrickson, Elli Schwarz, and Bill Farmer, emerged from our ideas and innovations program (i2).
We’re pleased to share details on this special project.
What is i2?
Clarity’s i2 program supports employee exploration and innovation providing resources, time, and team participation for concepts that could become realities. Each of us is encouraged to present ideas, receive feedback, work on a white paper or presentation, build a prototype, and see solutions in action.
In that spirit, three of our software engineers and NiFi experts worked together to create a conference session that provided a how-to with examples and demos showing basic capabilities of the data processing and distribution system.
NiFi in a Nutshell
Apache NiFi is an intuitive, scalable, and customizable dataflow system. It comes with an easy-to-use and appealing management UI, a large market of standard processors, and a vibrant Open Source Community supporting it.
“NiFi takes data from one place, processes it, routes it, and puts it somewhere else. It’s a sophisticated platform that allows you to be flexible in how you do that no matter how large
or small your dataflow.” – Ryan Hendrickson
Thanks to the Internet of Things, smart homes and cities, social media communications, and electronic financial transactions, among countless other digital phenomena, data drives almost everything. Remarkably, in one minute, Venmo completes $68,000 in peer-to-peer transactions, people produce 12 million text messages, and The Weather Channel receives 18 million forecast requests.* Demand for the capture, processing, enrichment, and delivery of useful data grows daily — even in the time it takes to read this post.
The Data Game (it’s challenging)
When processing data; its volume, velocity, variety, and value (the four V’s) are all considerations. Of course, software architecture questions must be answered as well: how is the data acquired, processed, stored, and enriched? In addition; how will analytics be run on the data and the results stored, how will analytics be run on the analytic result, and how will the results be served up on a web site? Answers to these questions help determine the dataflow stack. With its scalable and customizable dataflow system and a large market of standard processors, NiFi offers a flexible solution.
Features & Benefits of NiFi
Our team’s CodeMash session was designed to walk attendees through the NiFi basics by showing how to move data across servers and networks using the platform. It covered how to manipulate, enrich, and stream data through custom enrichment processors and in doing so, revealed the features and benefits of NiFi…
The Process is its Own Reward**
The advantages of using NiFi are many:
· Guaranteed Delivery
· Data Buffering with Back Pressure and Pressure Release
· Prioritized Queuing
· Flow Specific QoS (latency vs. throughput, loss tolerance, etc.)
· Visual Command and Control
· Flow Templates
· FlowFile Recovery
· Extensions (Processors and more)
· Classloader Isolation
· Scale Out (Clustering)
· Scale-up & down
· HTTPS & SSL
· Multi-tenant Authorization (User Mgmt)
“We’ve solved some pretty complex data flow and ETL challenges with Nifi. As we demonstrated with our CodeMash presentation, we were able to perform ETL mostly with built-in NiFi capabilities and used one simple custom data processor for a case where we needed a more custom solution.” — Elli Schwarz
To showcase NiFi’s capabilities and ease-of-use, our team ran some relatable demos. Using NiFi and implementing a variety of processors, the team polled reddit for new posts and aggregated subreddit topic names to serve up a simple web site that showed popular subreddits. They also used NiFi to consume tweets that were filtered on #CodeMash to light-up LED lights that were illuminated thanks to a Raspberry Pi that was equipped with MiNiFi.
“We were pleasantly surprised by the questions at the end of the session. People weren’t aware of NiFi’s capabilities.” — Bill Farmer
Clarity’s CodeMash 2019 Team
Ryan, Bill, and Elli collaborate on technical projects every day. Along with others on their team, they find great satisfaction in anticipating and meeting the mind-bending demands of our ever-changing world with custom software solutions. Smart, creative, and diligent, they run on mission and thrive on data science.
Ryan Hendrickson is a senior software engineer and technical project lead. He has been using NiFi for four years.
Bill Farmer is a principal software engineer, technical project lead, and Clarity’s vice president of engineering. He has been using NiFi for four years.
Elli Schwarz is a senior software engineer and lead technical contributor. He is an Elastic and NiFi guru. He pushes NiFi to its limits to solve complex challenges.
What’s Next for our NiFi Experts?
Ryan, Bill, and Elli plan to continue sharing their technical knowledge and insights. Catch them at upcoming tech user groups and conferences — follow us for announcements. They regularly attend Data Works MD Meetups in the Baltimore-Washington region.
If you’d like to request a copy of Clarity’s NiFi presentation from CodeMash 2019, please get in touch with us by email.
We discussed NiFi on The At Scale Podcast. Download it for an interesting talk about the technology — also available on iTunes and other podcasting apps.
January 2019 — we’re partnering with MongoDB to reinvigorate a local mongoDB users group. Our aim is to provide a place for developers to learn more about mongoDB, a non-relational, open source, document-oriented database.
Meetup discussions will focus on features, new releases, data modeling, schema design, and migration to mongoDB from other alternatives. Representatives from MongoDB; Clarity’s Senior Software Engineer and certified mongoDB professional, Mike Keith; and others will provide insights and share their experience.
MongoDB is the leading NoSQL database. Maybe you’d like to learn more about it? Join us! This group will be particularly interesting to anyone facing challenges with MySQL, Postgres, or Oracle options. Our meetup sessions will explore what mongoDB offers — a more flexible and scalable alternative.
We are collaborating with students at the University of Maryland’s iSchool in College Park. This fall, we outlined a specific business challenge for the students. To address it, they will use their information management skills to develop a Capstone project that provides possible solutions.
What is the Capstone Project?
During their final year of study in the iSchool’s undergraduate and graduate programs, students must complete a Capstone project. They choose from a list of real information management assignments provided by clients that have partnered with the University. Clarity is one of the organizations participating in this unique opportunity.
From the iSchool web site: Capstone projects “allow students to work through real information management challenges, providing them an opportunity to showcase their capabilities, synthesize their knowledge, and further develop their ability to communicate as information management professionals.”
Our Problem — Data Overwhelm
How do we better manage a wealth of proposal and contract information?
There is so much “data” to consider — detailed technical requirements, fluid management and staffing needs, budgets, prior teaming success, not to mention the complex business ecosystems involved.
As data scientists, we recognize the value of this rich information landscape. We’d like to dive into the data and efficiently use what it has to offer. Our primary aim is to navigate and analyze the possibilities of our contract-based business with more clarity and agility. As a result, we’ll make better-informed business development decisions. This capability becomes more critical as Clarity grows.
To solve our challenge, five undergraduate seniors will study specific requirements and research and report on ways to visually capture and analyze the complex and dynamic web of business proposal and contract information. Subsequently, two graduate students will further the project to determine how to develop and/or implement a solution. We are consulting with students in person and via email as they work to investigate, understand, and address our needs.
Next spring, students will present their final projects. As a small business, we see this partnership as an opportunity to gain creative outside input. “I’m excited to see what they come up with,” Clarity’s President, TJ Greenier, explained. “They may have different approaches that we haven’t considered.” At the conclusion of the project, we hope to use Capstone findings to improve our business development process. Our collaboration with students on a real-life business issue offers problem-solving practice and provides meaningful experience.
That’s Not All…
In addition to the Capstone initiative, our management team is conducting mock job interviews with iSchool students. We ask thoughtful questions and provide feedback on responses, promoting effective job seeking.
More on UMD iSchool
The University of Maryland College of Information Studies (iSchool) is located in College Park. It was founded in 1965. From the institution’s web site: From labs to libraries, we are combining principles of information science with cutting edge technology to foster access to information, improve information interfaces, and expand how information is used in government, education, business, social media, and more. The College is ranked in the top ten nationally by US News and World Report. Visit ischool.umd.edufor for more information.
Topic: Strengths of Relational Databases and NoSQL Solutions
So many data platforms, so little time… which one should you choose?
Data is everywhere in our lives and storage platforms continually evolve. Presented in March, TJ’s lecture provided an overview and comparison of relational databases and NoSQL solutions. Each has strengths, and choosing the right platform requires consideration of many variables. The talk shared strategies to use in determining the best platform. It provided a technical review of some common relational database options and included a discussion on transitioning to a NoSQL solution. TJ’s lecture covered key decision points in determining which platform is best depending on specific needs.
Through his presentation, TJ reviewed RDBMS options: Oracle, MySQL/MariaDB, and PostgreSQL and mentioned some of the NoSQL options, including MongoDB and ElasticSearch. He talked about RDBMS structure, schema, and transaction qualities. Next, he described how NoSQL allows for thinking about data differently and shared some of the benefits of the NoSQL database. Using MySQL and MongoDB, TJ demonstrated differences in uses of database platforms.
Students enjoyed hands-on examples and a question-and-answer period after TJ’s talk. TJ provided resume and job interviewing advice. The event was a success and an honor, and TJ is working with the iSchool to schedule another guest lecture — topic to be determined.
TJ Greenier has spent his career developing innovative software and technical management programs. He is an expert in agile software development, access and authorization, and auditing/compliance and has a passion for data and application usability. TJ has become an expert in data modeling, direct experience management and transitioning large legacy systems to modern data storage solutions. Most recently, he has worked to transition a large RDMS solution to a highly scalable and adaptable NoSQL solution, leveraging MongoDB and Elastic Stack.
TJ earned a B.S. degree in Computer Engineering from the University of Pittsburgh and his M.B.A. from Pitt at the Joseph M. Katz, Graduate School of Business where he was honored with the Dean’s Outstanding Student Service Award. In 2006, TJ founded Clarity Business Solutions — it became his fulltime occupation in 2011. When he’s not managing or coding, TJ enjoys skiing and mountain biking.