Creating a Data-Informed Culture – SF Product Talk Night Recap

“Data is ubiquitous technology. We’re constantly collecting it, manipulating it, and trying to monetize it.” – Jess Starr, Product Manager, Okta

Carbon Five San Francisco hosted Product Talk Night to discuss the different ways data and business intelligence can impact product development and business growth. Our guest speakers included: 

• Jess Starr, Product Manager, Okta | @okta
• Mira Cho, Product Manager, LaunchDarkly | @miracho

Kicking off the event, Jess Starr shared insights on how to build a data-informed culture, based on her experience with Okta’s consumer-facing reporting application and internal data platform. 

A few key takeaways: 

The challenges in creating a data-informed culture

Here are a few challenges you may encounter as a product manager, as you try to weave data into the culture of an organization that is new to data-driven approaches: 

• Defining Data DNA – Using data to support business decisions and initiatives — rather than collecting data without any particular purpose.

• Tooling – Choosing tools that are appropriate for the questions you’re asking today — and maybe also tomorrow.

• Data access and quality – Making sure the right people have easy-enough access to the data they need.

The Solutions

Process change – Process-change solutions can vary and will involve decision-making and buy-in from several stakeholders. Although cumbersome, process change is the first step teams must embrace to successfully build strong, data-driven products.

Build good products and do no harm – For companies working with user data, it is important to create and establish a data-informed culture that recognizes the responsibility we have to protect users’ sensitive data. We also need to check our own biases, around both data collection and analyses.

Mira Cho then shared learnings from her experience using data and experimentation to understand user behavior. 

How to get creative when you run experiments

“It’s not always easy to get access to data to make the best product decisions. Learning to experiment at LaunchDarkly and using feature flags to run tests has made us use our tool stack in more creative and flexible ways.” – Mira Cho, Product Manager, LaunchDarkly

Mira shared the tech stacks she has used as a Product Manager at 5 different companies and explained why each tool worked well — or didn’t — given each team’s specific needs.

Mira discussed how experimentation efforts vary widely for consumer and enterprise organizations, and talked about some of the tradeoffs. For example, as a PM, what can you do if you don’t have enough users to achieve statistical significance? A few options:

• Test higher upstream in the funnel, so you have more users. 
• Use qualitative information, in addition to quantitative data.
• Look for trends, even if they aren’t statistically significant.
• Measure metrics from back-end events, like mobile load speed or feature flag serve times, as a proxy for user activity.

Interested in attending our next event? Join the Carbon Five Meetup groups to receive updates about all upcoming Talk Nights and events hosted in our offices in San Francisco, Los Angeles, Chattanooga, and New York. 

---