Blog

+1 512 266 8271 


Wed, Jan 9, 2019, 2018 11:00 AM - 12:00 PM CST

Migrating a production Atlassian workload can be intimidating. The array of services, CFTs, containerization models, and other options within AWS make it even more challenging to know that you are making the right choice in your design. This webinar will look at the practices and common pitfalls encountered as teams move to AWS.

Wed, Nov 7, 2018, 2018 11:00 AM - 12:00 PM CST

Portfolio for Jira provides visibility of work breakdown and execution, estimation of project completion, and short-, mid-, and long-term scenario planning. While it is simple to use and integrate, even the experts like us have moments where we want to flip the table. Portfolio for Jira expert Amanda Babb will take us through some of the Common Behaviors of the Scheduling Algorithm and other Bad Behaviors which will have a critical impact on the use and adoption of Portfolio for Jira.  Whether you’re a current Portfolio for Jira user, or you’re evaluating it for your organization, you’ll want to attend this webinar to make the most out of your experience. 




By Bryan Robison

Our partners at Puppet and Splunk recently released the State of DevOps Report 2018 which mirrors many of the same observations that we see when working with our customers. DevOps isn't a sprint, it's a journey, depending on the organization's size, that can take organizations months or years to master. Instead of focusing on statistics like in prior years, this year's report introduces the five stages of a successful DevOps evolution and identifies and describes the practices successful, mature organizations have implemented during their journeys. The report has something to offer to organizations no matter where they are in their process. Even if your organization has employed DevOps for some time, you may be missing a critical component of your implementation.

Five stages in a DevOps evolution

The major takeaway from this year's report is that while there is no single path to a DevOps transformation, there are five foundational practices and five distinct stages of a successful DevOps evolution. Each stage is comprised of a set of practices broken down between "defining practices" and "contributors to success" which are identified and described for each stage of the process in the image below.

(State of DevOps Report 2018)

As organizations evolve from stage to stage, they refine and expand on the practices they implemented in prior stages. The report dives deep into the identification and makeup of each practice and describes how establishing practices identified as contributors to success in earlier stages such as "test infrastructure changes before deploying to production" in Stage 1 become mission critical defining or associated practices in later stages. DevOps is not a sprint, but a marathon, and requires organizations to have effectively executed each stage before advancing to the next one. Quite often we see that even the foundational practices such as monitoring, testing, and configuration management described in Stage 0 are missing from our customer's toolchains.

DevOps starts with a ripple

We frequently hear stories from our customers about how they began using Atlassian tools. These stores often begin with when a single team purchased a Cloud account or installed Jira onto a server sitting under someone's desk. Usage evolves and expands over time and eventually, the whole organization is reliant on the application for mission-critical operations. The report found that the DevOps evolution begins in much the same way: starting with a single team who defines their own practices for DevOps and shares them with other teams within the organization.

Where do I start?

The State of DevOps Report 2018 recommends starting with production by choosing an application with an, especially painful deployment process. We often describe DevOps as breaking down the wall between Development and Operations and deployments are where frictions between the two often occur. By automating deployments teams can improve collaboration and communication while reducing the friction caused when things go wrong. 

Since 2010 (before DevOps was called 'DevOps), Praecipio Consulting has been fortunate to work with customers at all stages of the DevOps evolution and guide them in using Jira, Confluence, Bitbucket, Bamboo, and other tools like Puppet, Splunk, New Relic, xMatters, qTest, and AWS as part of their technology standard. 

To learn more about how Praecipio Consulting can assist you with your DevOps journey, be sure to register for our upcoming webinar "DevOps: An Interpersonal Approach" on October 10!

Wednesday, October 17

8:30 a.m. - 12:30 p.m. ET

Top of the Town


Want to uncover how the most effective government IT teams leverage Jira Service Desk? Attend our Jira Service Desk workshop on October 17 to discover how to automate your agency's IT service management (ITSM) process and achieve record-breaking mean-time-to-resolution.

Register for this complimentary hands-on experience to overcome the three big inhibitors IT teams face in reducing downtime: 

  • Working in multiple systems
  • Inability to communicate and demonstrate visibility across teams
  • Difficulty managing alert overload during outages

Attendees will interact as teams in a simulated service desk environment and respond to incident scenarios with a streamlined ITSM toolchain, including Atlassian's Jira Service Desk.

Complimentary breakfast and lunch will be provided. Check out our agenda.




By Michael Kelly

As with life, the only constant in DevOps is change. To position your teams to seamlessly flow with the changes and to empower them to innovate, some of the old ideas on Operations and Development practices must be left behind. An integral component of DevOps is culture. An iterative approach to a collective ownership should be taken when planning your move toward a DevOps environment. You can start your DevOps journey by advocating for the adoption of a consolidation of tools designed specifically for DevOps, which provides an environment of transparency and ease of use.

Agile

Adopting Agile development practices, iterative planning and short cycles can alleviate the frustrations that fester as ill-planned deadlines are not met. Developers will be more empowered to meet shorter competitive deadlines while working appropriately planned sprints and have more time to innovate and experiment. Agile development will also be far less likely to produce bugs that are difficult to locate and fix.

Continuous Integration

Adopting the use of CI, you can avoid code that may have to be completely redone when it fails in production. Tests performed on small changes to the code, committed several times a day, have proven to be less prone to failure. Smaller changes lead to higher quality, have less risk, and allow for easier code reviews and locating problems.


Infrastructure as Code to define test, prod, and other environments has proven very successful. Self-service infrastructure for developers to have provisioned, tested against, and disposed of, can bring your team to a place of continuous learning and experimentation and make reactive scrambling to determine what is broken a thing of the past. Utilizing these methods, environments can stay in closer parity with fewer differences and remove the surprises when tests pass on the test environment, but fail in production deployments, as tests are more complete and valid.

Monitoring

Monitoring infrastructure, operating systems, application, and third-party cloud services are crucial in observing trends, understanding the health, and receiving an important feedback loop. Tying all of the monitoring together with a tool specifically designed to do so, and having alerts visible to everyone involved will provide transparency, trends to act proactively on, and ensure that the end user is receiving the highest quality product.

ChatOps

ChatOps is not the same as IM, or instant messaging. ChatOps tools have the capability to integrate with other tools. Seamless integration makes statuses of all the components transparent at all times. Utilizing ChatOps in your DevOps environment provides faster and continuous communication at all phases, from planning to a commit to a production deployment and its effect on the end users.  

Continuous Communication

Select tools that enable continuous communication.

Feedback loops should be real time and visible to teams at all times.

To consider your environment DevOps, you should be in a state of constant and iterative improvementAgile practices, CI, Monitoring, ChatOps, and the flow of Continuous Communication are all necessary steps to reach this goal.

If you're still interested in our approach to DevOps, be sure to register for our upcoming webinar "DevOps: An Interpersonal Approach" on October 10!


By Chris Hofbauer

Leveraging Atlassian's dashboards and gadgets can provide teams within an organization the visibility and transparency into their work they may be lacking. The use of these tools gives greater insight into work in progress and completed work for individual teams or team members as well as providing top-level views of all the work across teams. Dashboards can be configured in many ways and be custom tailored to surface whatever information is desired. Dashboards are made up of gadgets as they are the driving force behind the data. These gadgets are embedded into the Dashboards and the information within them is determined by the JQL in the filters.

Some of the commonly used gadgets are the Pie Chart, Jira Road Map, Average Age Chart, Created vs Resolved, and Issues in Progress. Below is an explanation of these gadgets and their uses.

Pie Chart Gadget: The Pie Chart gadget can display data from many fields. The determination of which field you choose depends on your particular use case. Assignee, Resolution, Status, and Priority are some of the more powerful and frequently chosen fields. The issues are split by these fields and displayed in a manner that is easily digested as a visual. Clicking on any chunk of the pie in the gadget will take you to a more comprehensive list view of those issues. From here the data can be dug into deeper or exported for reporting.


Created vs. Resolved Gadget: The Created vs. Resolved gadget is exactly how it sounds. The data in this gadget will show all the issues that are resolved against those that are not. The use of this help to track how well the team is keeping up with their work items. Team leaders can view this data to determine where they are within a given sprint. If things are going as planned, the later they are in the sprint, the fewer number of issues should be logged. Since the Created vs. Resolved gadget focuses on work submitted and work completed, it is a good choice for Kanban teams to see their workload and progress made within a given timeframe.


Issues in Progress Gadget: The Issues in Progress gadgets provides a more granular view of all the issues within the development cycle that are still being worked by the team. Team leaders can use this information to get a more detailed view and quickly determine if there are some bigger issues in progress that could not be completed before the end of the sprint.


Jira Road Map Gadget: Development teams will get great use from the road map gadget. This gadget shows what has been completed within the sprint and what versions are due to be released at the conclusion of the sprint. Teams leaders can leverage this information to see where they are at in their projects and how their teams are performing.


These are only some of the many gadgets that can be used to provide the desired visibility and transparency into the work in progress. Team leaders will find these gadgets easy to use and customizable to whichever way fits their needs. It is important for teams to leverage these dashboards and gadgets so that their work can be done as efficiently as possible and completed in a timely manner.


By Chris Hofbauer

For organizations that use the Atlassian product suite, it's critical to decide whether to administer the products in-house. After all, there are significant considerations with infrastructure, networking, product tuning, let alone the configuration of the products. Choosing to engage in a Managed Services agreement can be extremely beneficial. A great Managed Services team can be a great asset and provide tremendous value to any size organization.


By Chris Hofbauer

For organizations that use the Atlassian product suite, it's critical to decide whether to administer the products in-house. After all, there are significant considerations with infrastructure, networking, product tuning, let alone the configuration of the products. Choosing to engage in a Managed Services agreement can be extremely beneficial. A great Managed Services team can be a great asset and provide tremendous value to any size organization.

One of the greatest values provided is the administrative relief you get from a Managed Services team. The Atlassian products require devoted attention, especially within organizations that use it for all sources of truth. Having a Managed Services team allows an organization to have fewer full-time employees dedicated to the tools and instead on their primary area of expertise. Depending on the contract, signing onto Managed Services is typically half the cost of hiring on a full-time Atlassian administrator as well.

System monitoring, performance, and upkeep can also become a major burden to bear. Atlassian products and add-ons are frequently updated and if the applications are not well kept, the system can become outdated fairly quickly. With a Managed Services team, we work closely to ensure performance and maintenance doesn't fall behind so teams can continue to leverage the power of the Atlassian toolset.

One of the most valuable aspects of hiring a Managed Services team is the tribal knowledge and best practices that come along with it. When an organization has one or few Atlassian administrators, they are limited to the knowledge of those individuals. With a Managed Services team, you have the entire company to leverage. We have a large team that has seen various installations and configurations (both supported and unsupported) of the Atlassian tools, including those we host in Cumulus, our Atlassian Hybrid environment. By seeing all these variations, we are able to provide the best solutions that have been tested and proven.

The Atlassian product suite requires attention and care from people who are dedicated to the tools. Our Managed Services team helps to relieve administrative overhead for every organization through system performance and monitoring as well as best practices through tribal knowledge, all at less expensive option compared to a full-time employee or team. With this, hiring a Managed Service team provides peace of mind and great value.

Older Blog Posts

By Michael Knight

You've probably heard about a lot of the benefits DevOps teams enjoy - more effective investments, less stressful deployments, increased collaboration and visibility, and a healthier, happier, more empowered team.  With such encouraging results, the choice to take on a DevOps approach becomes an easy one. The trickier question, then, is what products can help your team take on that approach?

Fortunately, there are thousands of applications to help get you there.

Unfortunately, there are thousands of applications to help get you there.

We've worked with hundreds of clients across virtually every industry, and we have encountered untold numbers of applications, tools, and solutions along the way. In our experience, the Atlassian stack is a top choice.

We typically see a lot of added value with each team using an Atlassian stack:

Cost-effective

The overall solution is more cost effective. Atlassian prefers to spend money on product development, rather than supporting a gigantic sales team. This enables them to build best-in-class products while keeping the price tag favorable.

Integrations

Every application in the solution is integrated. Again unlike other companies, Atlassian produces products across the entire DevOps infinity loop, which results in a number of standalone products that integrate extremely well. It's kind of like the days before Apple became a dongle company when all of their products just worked together.

Customizable

Teams can customize the products to meet their needs. Not all teams want to work the same way. Differences as large as Scrum vs. Kanban or as small as where to record Acceptance Criteria can be easily managed.

Numerous applications

The Atlassian marketplace has over 1,700 different add-ons, meaning there are options to extend into nearly any other existing application in the DevOps space. If that somehow doesn't cut it, there's also middleware like Workato to help bring systems together.

Atlassian prefers to focus on building products that people love, and we've seen and confirmed for a dozen years that teams love using the products. And after all, isn't empowering teams what DevOps is all about? 

Older Blog Posts

Wednesday, October 10, 2018 11:00 AM - 12:00 PM CST

Learn how to take a more interpersonal approach to the DevOps journey

DevOps is a journey. Often, companies will place a strong emphasis on tools, automation, and making big changes - fast. To start the DevOps journey, teams don't need to begin with the tools or automation, or even making big changes. The key is to start small and with an interpersonal approach. Chief Technology Officer, Christopher Pepe, will take us through a more interpersonal approach to DevOps, and how teams can begin their DevOps journey today.




By Christopher Pepe

Walking the halls of Praecipio Consulting you will often hear people talking about the cost of quality. One of the areas that we focus on with our clients is the cost of rework which is due to poor quality and can often be improved with a better process. While there are many reasons for poor quality, a common reason (which permeates all aspects of our lives) is rushing. One of my favorite YouTubers put it as "Never rush that which must be done quickly." You might ask what sitting in snow, or a survival skills/mindfulness youtube channel has to with the quality software, incident management, or any of the other high tech stuff we do. Especially when it comes to high stress, time-sensitive situations there is a lot to learn from other domains.


By Christopher Pepe

Walking the halls of Praecipio Consulting you will often hear people talking about the cost of quality. One of the areas that we focus on with our clients is the cost of rework which is due to poor quality and can often be improved with better process. While there are many reasons for poor quality, a common reason (which permeates all aspects of our lives) is rushing. One of my favorite youtubers put it as "Never rush that which must be done quickly." You might ask what sitting in snow, or a survival skills/mindfulness youtube channel has to with the quality software, incident management, or any of the other high tech stuff we do. Especially when it comes to high stress, time-sensitive situations there is a lot to learn from other domains.

I was a waiter for about a month. More correctly, I was a terrible waiter for about a month. I never got orders right, I failed to bring second drinks, or really fulfill most customer requests satisfactorily on a busy night. I was ok at serving lunch and slow nights but I was easily overwhelmed by the volume of tasks that waitstaff has to manage. I often look back at that time for insight because I have generally been pretty good at most jobs but failed so completely at waiting tables. My core issue was that I was also rushing because I was in a hurry and that lead to decreased quality of service and longer cycle times. Because I rushed: I delivered the wrong dishes (or accidentally took dishes meant for other tables causing the rest of my team to fall behind too), I forgot that coke, I didn't split your check, I made the wrong change... And with each mistake, I had to stop, correct that problem, and leave my other tables with a worse experience too.

You can go a lot faster if you start with slowing down.

Older Blog Posts

At the 2018 Atlassian EuroSummit, Praecipio Consulting wins an Atlassian Partner of the Year award for the fourth consecutive year

For Immediate Release
09/3/2018

BARCELONA, September 3, 2018 -- Atlassian announced today that Praecipio Consulting has received Atlassian Partner of the Year 2018: ITSM for their outstanding contribution and achievements during Atlassian's fiscal year 2018. This includes exceptional efforts in developing new business, thought leadership, and products and services that complement Atlassian.

"It's a privilege to be named Atlassian Partner of the Year for the fourth year in a row, and it is a milestone in our successful partnership with Atlassian," Praecipio Consulting Founding Partner Christian Lane said. "Serving as an Atlassian Platinum Solution Partner is a critical and core capability and we are honored to be named an Atlassian Partner of the Year for the fourth year in a row. We look forward to continued success and to bringing best-in-class solutions to our customers contributing to their success and achievements. Winning in the IT Service Management category is a validation of the expertise we have in the ITIL-based ITSM disciplines. Our business' roots are in ITSM."

Praecipio Consulting was one of 15 recipients honored as Partner of the Year during the Atlassian Partner Day at the company's European Summit in Barcelona, Spain. 

"Atlassian is thrilled to recognize and honor our 2018 Partner Award recipients", said Martin Musierowicz, Atlassian's Head of Global Channels. "Solution Partners are instrumental to our customers' success and we are excited to be able to highlight some of our top partners who are going above and beyond to support customers and provide Atlassian services."


Praecipio Consulting enables impactful transformations to build, release, run and manage better products and services. As a cross-disciplinary consulting practice, Praecipio Consulting specializes in DevOps, ITIL-based ITSM processes, Agile/SAFe, and various ALM frameworks. Praecipio Consulting is a trusted partner to clients of all sizes and industries.

Older Blog Posts


By Elizabeth Moreno

In an age where there’s a tool for every toil, we’re fortunate at Praecipio Consulting to be able to work with the best tools available. As an Executive Assistant who survives on staying organized, ahead of the game, and juggling with an exceptional attitude is just part of the day-to-day. If you hear “How do you do it?” you are not alone. How do I do it? The right tools.

Finding tools with exhaustive capabilities, that are simple enough a non-technical user to navigate, isn't an easy task. So here are my top three tools that help me find the most success in my day-to-day.

Calendars 

I live, eat, sleep, and breathe with my calendar - ok not that extreme. But, our team does use Google calendar, and from your first day, the expectation is that it’s up-to-date and detailed enough that others will have the information needed in order to make scheduling meetings with you a breeze. Establishing this as a best practice will help teams maintain prioritization, eliminate duplicity, and create a more cohesive dynamic.

Task Management 

Jira serves as our task management tool. I will either assign tasks to myself and/or manage tasks for other projects or teams. Jira is an Atlassian workflow management tool that helps teams plan, track, and manage tasks, tasks within larger tasks, and prioritize projects while eliminating the common silos and blockers associated with project management.

Communication

Team chat and collaboration tools are a necessity for productive teams and there are a couple of good options out there to choose from. Our team currently uses Stride, an Atlassian chat platform that facilitates a more communicative environment; however, we will be moving to Slack soon. Stride helps our entire (local and remote) team stay connected, engaged, and informed. Several of our team members are constantly on the road traveling to visit clients, so having a communications tool is very important.

Ultimately, no one wants to drop the ball, fall behind, or be so overwhelmed they have a case of the Mondays every day. So use the tools you have, and make sure you are maximizing the tool's capabilities. If you aren't sure that you have the right tools to be successful, pioneer the charge for change and introduce one of the tools I mentioned. Who knows, it just might work! We all want to get the job done and sometimes the path to completion can be challenging, so why not make that path smoother for the next person or the next trip? Pick a tool, put it in place, and practice good process.

By Brian Nye

While service desk agents do everything they can to avoid firefighting, they are often focused on extinguishing one fire and moving to the next. This usually causes tickets to smolder in some status of "not quite done" until months later when they will finally be closed out (thanks bulk edit!). The good news: there is a way to keep things moving using out-of-the-box functionality. No longer will your metrics be inaccurate because people aren't "moving their tickets through the system." Jira Service Desk can help do the moving for you with automation.

Putting out Smoldering tickets

Many workflows offer customers a chance to review the ticket before closing. But, replying to the work request isn't always the top priority of the customer, which in turn, leaves the ticket to smolder in an almost done state. Instead, Jira Service Desk can help you do a fully extinguish the request by doing a couple of things, messaging the customer on impending closure and auto closing the ticket with no response. Just follow these steps below.

Step 1: Create SLAs

While this may seem odd, SLAs can be used for more than just metrics, they are a great trigger for automations due to the extended functionality SLAs bring in Jira Service Desk. Start by creating two SLAs, call them Time in Resolved - Customer Notification and Time in Resolved. Set Time in Resolved - Customer Notification to the parameters shown in the screenshot below. Note, the SLA time can be changed depending on the amount of time you want to elapse before notifying the customer that their ticket will be closed. The SLA for Time in Resolved will have the same start and stop conditions, but put the goal time to be more than the goal of the notification trigger (for example, if the notification is set to send 120 hours after entering the status, than set the goal for the auto close to be 168 hours as this will give 48 hours for the customer to respond).

Step 2: Create Jira Service Desk Automations

Great, now that these SLAs are in place, let's use them to trigger Jira Service Desk Automations.  

Step 2a: Time in Resolved - Customer Notification

For this Automation, you will want to set the When to trigger off of the Time in Resolved - Customer Notification SLA when the SLA has been breached. Feel free to add an optional If statement should there be situations in which the SLA should not be executed. Lastly use the Public Comment option for the Then statement to send a message that the customer will receive. Included is a screenshot of this automation.

Step 2b: Auto Close Resolved Ticket

For this Automation, you will want to set the When to trigger off of the Time in Resolved SLA when the SLA has been breached. Feel free to add an optional If statement should there be situations in which the SLA should not be executed. Lastly use the Transition Issue option for the Then statement to move the issue to the final status. Note that it is best to use a hidden transition which does not require any fields or info as this is done through an automation. Included is a screenshot of this automation.

Step 3: Find other small fires to put out using automations

This is just one example of how automations can be used to keep customers engaged on the ticket and closing out issues that have been resolved. This same logic can be applied to many different areas in Jira Service Desk and can keep your front line firefighters focused on the hot spots and less time doing clean up!

If you still want to learn more about Jira Service Desk automations in action, join us for our next webinar on September 12, 11 a.m. CST: Automation with Jira Service Desk.


By Morgan Folsom

One of the most powerful integrations in the Atlassian ecosystem is the native link between Jira and Confluence. For users working in both tools, the transition can be seamless if you do it right, but clunky if you don't. 

Now, what if I told you there was just one Confluence macro you could start using today that will immediately make reporting in Confluence easier and help you (and your team) keep track of your work? The Jira Issues macro is the go-to when reporting in Confluence.

Here are some tips to get your team to live their Atlassian life-to-the-fullest.

Insert an issue count for a Jira filter

Let's start small. Insert a link to Jira with the number of issues returned from a Jira Query Language (JQL) query.

This is useful to pull up basic metrics for a high-level overview. The macro becomes a link to the filter, so if you want to review the issues in-depth, you can quickly hop over to Jira's issue navigator. The table below is an example of how our marketing team tracks employee blog post submissions.

To insert an issue count:

  1. Insert the Jira Macro
    1. Select the  in the top menu bar and select Jira Issue/Filter, OR
    2. Type { on your Confluence page, search and select Jira
  2. Enter in your JQL query
    1. To input an existing filter, type "filter = "Filter name", OR
    2. Type in the JQL directly
    3. Be sure to click on the Magnifying glass to execute the query
  3. Select 'Display Options' at the bottom of the dialog box to expand the options.
  4. Select 'Total issue count'
  5. Click Insert, and Voila!

Insert a single issue into Confluence

This macro can also link to a single Jira issue to a Confluence page. That means not only can you see what issues are important (and what status they're in) in your documentation, but you can also see who's talking about the issue when you're in Jira.

Take, for example, this blog post. My progress is tracked on a Jira issue, linked to this very page in Confluence. Below you can see how it looks on the Confluence page I'm writing in. 


If I click on that link, I'll move over to Jira where I can see all of pages in which the issue has been mentioned under Issue Links. Right off the bat, I can see that the issue has been mentioned on this page as well as another tracking Blog Content. 


To insert one issue:

  1. Insert the Jira Macro and enter in your query (steps 1 and 2 above)
  2. Select one issue from the list
    1. If you know exactly which issue, you can simply type the Issue Key into the search bar and hit enter. 
  3. Expand the Display Options and select 'Single Issue'
  4. Select 'Insert'

Use the Jira macro to insert a list of issues in a page in Confluence

Remember that filter you entered in above? You can insert that filter into your page, too. Filters inserted with this macro are dynamic - that is, as the issues are updated in Jira, the Confluence page will reflect the most up-to-date information. You can customize which columns appear in the macro just like you can in Jira. To head into Jira, you can select the individual issues, or click on the total number at the bottom ('2 issues') to pull up the query in Jira.

 

To insert a filter:

  1. Insert the Jira Macro and enter in your query (steps 1 and 2 above)
  2. Expand the Display options and select 'Table' 
  3. Edit the maximum issues and columns to display.
  4. Select 'Insert' to add to the page!

Create a Jira Issue from a Confluence page

If your issues don't exist in Jira yet, don't worry. This macro can create new issues in Jira if inspiration hits while you're editing a Confluence page. The issue will be created and you won't even have to leave the page. 


Additionally, you can also create issues from Confluence while viewing a page - simply highlight some text and then click on the Jira icon that appears.

  1. Insert the Jira Issue Macro
  2. Select 'Create New Issue' on the left panel
  3. Complete the form
  4. Select 'Insert'

This one macro can solve many of your reporting needs in Confluence. What's more, you can provide context around the data instead of just straight data. The Jira Macro is a great way to keep team members informed without navigating from Confluence to Jira and back again. 


By Christopher Pepe, Dragon of the West

What, why?

My favorite class in college was Neural networks for non-linear control systems which was way out of my league but I wanted it, so I powered through. I majored in mechanical engineering and studied computer science and electrical engineering because of a passion for robotics. This class blew my mind. Life being what it is, I went off to build a career which took me away from machine learning until recently. Things have come a long way in the intervening years and I wanted to recreate the approach we used in this class.

The animations that we created were what most captured my imagination because they would show the controller learning and improving.

My old code is unreadable. Between my decade and a half of improved coding style, and the early NN libraries that we used, I didn't glean much what code I still have. I spent a while trying to untangle that mess before just doing it from memory and filling in the gaps with frog DNA.

The experiment

I created a simple universe in which a rocket that can only move along the Y-axis. It is given the task of moving from its current position to a goal state along a "minimum jerk based trajectory." The rocket has a single engine that can fire downward. Since it has a mass of 1000kg it needs to produce a thrust of 9800N to hover (I think, it has been a while). Anything less than that and the rocket will lose altitude, anything more and it will gain altitude.

This required some serious way back machining to build the physics engine which tracks the current position, velocity, and acceleration of the rocket for a given timestep. Acceleration is calculated and velocity, and position are integrated from there.

def updateState(self):
        self.time += timeStep
        #calculate the current acceleration based on thrust assuming it is instantaneously applied
        self.accelerationY = (self.thrust/self.mass) + gravity
        #calculate the speed based on acceleration assuming it has instantaneously changes and is constant
        # (we could += here but it makes the physics harder to follow)
        self.speedY = 2*self.accelerationY*timeStep + self.speedY
        #calculate the change in height given our current state assuming everything is constant over the time step
        ##NOTE: oddly self.accelerationY**timeStep creates an imaginary number
        newY = self.accelerationY*timeStep*timeStep + self.speedY*timeStep + self.y
        #the ground is 0
        if newY >=0:
            self.y = newY
        else:
            self.y = 0
            self.speedY = 0

While this approach requires iterating over time, this update method is an easy way to measure the effect of changing the thrust over the timeSteps. Any thrust profile can be plugged in to see what the rocket would do. For example, one could leave the thruster off of 2s and then apply 10,000N of thrust. In that scenario, the rocket would free fall for 2s and then continue to fall before overcoming gravity and gaining altitude.

The code below was used to allow the trained neural network controller to fly the rocket and try to achieve its goal. Given the current state (time to goal, position, distance to goal) the network outputs a thrust to apply for the next time step. Because everything in the neural network is normalized we have to scale the output via hugeify(). My rocket is called dragon too.

thrust_normal = dragon.controller.predict( state )
dragon.thrust = hugeify(float(thrust_normal), thrust_min, thrust_max )
dragon.updateState()
state = dragon.getState()


Results

Naturally, I tuned all of the hyperparameters for the best results. I learned a bit about why this network performs well with those parameters. I again found that it took a surprisingly few number of neurons to achieve the goal, and more neurons generally didn't improve things. Increasing the batch size improved the quality of the output (I assume because the network could see more of the time series in each training epoch). The dramatic change in learning takes places around 19 epochs, and beyond 100 epochs there is no real improvement in learning (presumably overfitting at this point).

The path is simple. Starting at a height of 15m, moving at 0 m/s, fly up to 50m over the course of 45s. The minimum jerk trajectory to achieve that goal looks like this

and after 100 epochs of training the neural network produced this trajectory like this

What I trained on is the thrust for a given state (time to goal, position, distance to goal) and the thrust curve doesn't look as nice (by picking and choosing it can look better). The left is my calculated minimum jerk trajectory, the right is the output of the neural controller.


Here is an overly simplified animation of the resultant trajectories over a number of training epochs. Each path briefly flashes to the screen before the animation plays. Each marker remains at its final position so you can see how each run compares.

If you are curious, here is the output from the network after 100 epochs. The code is here. This was our approach in the aforementioned class. Generate tons of positional data and then animate it rather than have the simulation and controller working together in real time. I plan to move to a framework like OpenAI's Gym for future work.


Improvements

For the animation, I wanted to use an actual rocket looking figure, and scale fire coming out of the nozzle proportional to the magnitude of thrust but will tackle better animation next time.

I really wanted to approach this problem with a recurrent neural network since it is a time series problem as presented. I was unable to figure that out in the time I had for this effort but will revisit it. I think an approach like a text generator would work but need to noodle through some of the challenges unique to this problem.

Finally, I was happy to have a network that output a thrust rather than something like an acceleration or displacement but how I got there was weak. Generating training data requires solving for the physics to determine the "correct" thrust and this is flawed in a few ways:

  • if we can perfectly model the system we don't need a NN controller to try to learn the dynamics
  • this system cannot handle the dynamics changing (e.g. mass changing from burning fuel) or a hundred geese landing on the rockets nose

Another area of interest is replacing the trajectory generator with yet another neural network. My current thinking is that I can use a Generative Adversarial Network (GAN) to train a generator on the trajectory creation skill and feed in actual minimum jerk trajectories to the discriminator. There are still problems with this approach but it will be worth the exercise if only to build a successful GAN.

Anyway, another fun toy project completed.