Category Archives: Sales Matters

Posts in this category relate to activities and operations intended to increase demand for what an organization offers its customers.

News, Operations, Sales Matters

Prompt Engineering: How We Put Generative AI to Work For Our Business

1 Comment

Summary:

In just six prompt engineering iterations we achieved real business value with generative Artificial Intelligence in that we are now able to:

  • Significantly reduce the time required to review and assess client submissions using our WHAT-WHO-WHY rubric.
  • Provide a comprehensive evaluation and quality suggestions for each entry.
  • Make explicit and accessible our assessment and recommendations logic that had not previously been made clear to those we work with.
  • Increase our capacity to provide service.
  • Quickly and easily incorporate new data and insights into our process.

While we found it doable, prompt engineering required careful attention to detail and follow-through to identify and address what worked and what did not work at each step.

Now we have a fantastic starting point from which we can easily apply our facilitation skills to deliver even higher levels of value to leadership teams.

Background

Large Language Models such as Bing and Bard have received a lot of attention for their ability to chat intelligently (or so it seems!) about almost anything. However, there are not yet many examples of chatbots providing real business value.

At IntelliVen, we wanted to test generative AI’s ability to improve the quality of our work while simultaneously lowering costs. We are pleased to have found that chatbots can generate significant business value.

In this post, our goal is to share how we achieved business value using generative AI so that readers can build on our efforts and push our thinking (and their own thinking!) even further in this rapidly evolving discipline.

The key to what we achieved is Prompt Engineering, an iterative process by which we created a suitable and reusable prompt for our use case.

Prompt Engineering Definition

Prompt engineering is the process of designing and refining prompts to improve the performance of AI models. It involves techniques like using the right words, format, length, and parameters to coax the best performance from an AI model given its training data.

Prompt engineering can also include providing pertinent background, such as:

  • The profile of the person inputting content.
  • What perspective to take when assessing input.
  • The profile of persons who will read the output.
  • What types of output are requested.
  • What indicates high quality output.

Especially when you plan to interact with a chatbot for the same purpose on a recurring basis, it is critical to invest in prompt engineering to ensure you get the highest value results.

Use Case

At Intelliven, we help leaders and their teams architect, build, govern, and change their organizations. The cornerstone to our approach is to first align leaders and their teams on the definition of their business; that is:

  • WHAT the organization provides.
  • WHO buys what they provide.
  • WHY buyers choose to purchase from the company.

PREPARATION: We first ask the leader and their top team to each independently fill out the W-W-W template with their perspective on the three dimensions that define their business. We assess individual responses for clarity and specificity and then compare responses to identify what is common and what is different between them.

For every submission we:

  • Compare each response against a rubric that outlines what each element of the W-W-W should and should not contain to come up with at least three helpful pieces of feedback.
  • Analyze the frequency of terms used to identify differences and similarities across submissions.

FACILITATION:  Next we facilitate a session (or sessions) in which the leader collaborates with their team to align on what should and should not be included in a clear and concise definition of their business which then serves as input to virtually every aspect of their business.

It takes a consultant with significant expertise (of which there are but a few) about a half-hour per entry to internalize submitted input, apply their best thinking, and document their assessment of each submission. Depending on the size of the group, this process may take up to half-a-day per organization.

Prompt Evolution

Our goal was to see how much AI could help us with preparation, which includes intake, assessment, recommendations, and documentation for each entry. The following table summarizes six iterations of prompt engineering towards this end:

Value Delivered

We found that generative AI effectively added value to all four preparation steps (intake assessment, recommendations, and documentation), AND generated better and more comprehensive results; and it did so almost instantly.

Specifically, generative AI:

  • Makes the logic we previously used intuitively explicit for our consultants and clients. In other words, the chatbot explains why it made the assessment and suggestions it did.
  • Saves time and cost by dramatically reducing the effort required to assess input and document results.
  • Produces a high-quality guide our consultants can use to facilitate the collaborative process with leadership teams to reach alignment on a consolidated version.
  • Comprehensively applies all facets of our assessment rubric to every submission with no additional cost or effort. Previously we were content to stop after identifying just one or two salient points!
  • Enables us to present assessments and suggestions with more friendly and readily received language than we tend to draft on our own.
  • Integrates with the full comprehensive set of up-to-date world knowledge in its suggestions when we use a bot that is connected to the internet and pointed to content on our site.
  • Easily updates response logic with new case data or when we update the assessment rubric.

Note that, even though it has great value, we do not plan to sell our prompt or to charge for its use. Rather we have integrated it into the W-W-W method to:

  • Enhance the quality of our assessment, suggestions, and facilitation.
  • Increase the number of people able to do what our best do.
  • Dramatically lower the time and cost to do what we do.
  • Increase the attractiveness of our offering to prospects.

Considerations

While AI provides great value, it is important to:

  • Know when to stop. When a prompt is activated more than a couple of iterations on a given input, the chatbot’s responses tend to go in circles that add little-to-no additional value.
  • Check everything carefully. Many times generative AI makes up content which we have learned to interpret as indicating that “something along these lines is needed here”. For example it will make up a role in an organization for the buyer when the submitted entry does not identify one.
  • Never take the chatbot’s output as a final answer. Use it simply as quality input to the collaborative process.
  • Note that different chatbots give wildly different responses to the same prompt and the same bot is apt to give a different response to a resubmission. While initially concerning, we learned to consider each response as one more round of input for collaborators to consider.
  • Chatbot responses offer little in the way of creativity or imagination except when it makes things up instead of pointing out gaps. Responses are simply a complete and comprehensive application of facts and rules. Creativity and imagination come from the leader and team … not the machine!

Next Steps

We are preparing to share soon the results of using our AI-powered WHAT-WHO-WHY assessment and recommendation engine on a real case study.

To try out our engine for yourself:

Fill out and submit a WHAT-WHO-WHY Template to run your case through our process!

Additional Resources

Finance Matters, Operations, Sales Matters

Diligence Support for Federal Sector

Leave a comment

If you are looking for opportunities to invest in firms that provide professional services and software to the U.S. federal government, you know how challenging it can be to assess the potential and risks of these businesses.

Working with the federal government is not just another market, it is more like a different planet. Everything is different. The language is different, money is different, and no one can tell anyone else what to do.

Mastering the art of developing coalitions of support is vital. In short, you need a partner who understands the unique dynamics and complexities of the federal market, and who can help you make informed and confident decisions.

Approach

That’s where IntelliVen Federal Diligence Support comes in. We are a team of senior operating partners with deep experience and insights in the federal sector, and we offer a range of services to help you evaluate and validate your investment targets, identify value creation drivers and how to activate them, and help you craft a compelling investment thesis.

Whether you need just a few questions answered, a comprehensive Go-to-Market diagnostic, a competitive landscape assessment, or a strategy roadmap, we can provide you with the data and guidance you need to make the best indication of interest or purchase offer.

We have worked with dozens of public, private, private equity owned, and venture capital backed companies in the federal space, and we have a proven track record of delivering high-quality results on or ahead of schedule and on, and usually well in excess, of targets.

Next Step

We know how to navigate the regulatory, contractual, and operational challenges of the federal sector, and we can help you identify and mitigate potential issues and risks. Our network of federal experts, thought leaders, and former government executives provide valuable insights and connections.

IntelliVen is not just another consulting firm. We are a partner who shares your vision and goals, and who can help you achieve them faster and more effectively. We are driven by our passion for helping leaders and their teams architect, build, govern, and change their organizations for breakthrough improvements in performance and growth.

We have developed a system of tools, workstreams, and tutorials based on what we have learned from running and growing dozens of successful organizations. We call it the Manage to Lead (MtL) System, and we use it to help owners, leaders, and teams to get clear about what they seek to accomplish; aligned on their purpose, roles, processes, and metrics; and on track to perform better and grow faster.

Contact IntelliVen founder and Managing Partner Peter DiGiammarino to learn how IntelliVen can help with your diligence needs in the federal market.

See Also

Compusearch Case Example

Operations, Product Development, Sales Matters

4 Steps to Engaging Communities to Build Better Products

1 Comment

Tech companies constantly need to decide which products and features they should develop next to drive the most value for users. Listening to and elevating the Voice of the Customer through a community-based approach is an ideal way to inform their decision-making, especially when better meeting current user needs accelerates value.

As David Spinks shares in his SPACES framework he developed with the CMX Team, communities are valuable to host organizations because they provide Support to members, input to Product development, Acquisition of new customers, Contribution to best practices, Engagement in the brand, and help each other Successfully derive maximum benefit from company products. 

Double-clicking on the goal to inform Product development, this post provides a guide for how to thoughtfully activate a community to decide which products and features to develop next.

The key to success is strategically connecting product team leaders and relevant community members so that leaders better understand user pain-points and commit to addressing them. 

For maximum impact, follow these four steps: Set your strategy, Prepare the Conversation, Hold the Event and Follow Up:

I. Set Your Strategy

  • Identify Top User Pain-Points: Assess the input collected from community members to determine the pain-points that keep users from achieving their goals. Scaled research is a good way to identify the most important pain points. I am sure there are AI tools that can help you analyze large quantities of data and pull up keywords and phrases that appear as challenges for your users. Let me know which ones you recommend!
  • Determine which Pain-Points to Address in Conversation: Share collected pain-points with relevant product managers to identify which are related to work already in progress and which are yet to be roadmapped. Top pain-points that are not already set to be addressed, or that are on the roadmap but not prioritized, are candidates for advocating on behalf of users to accelerate product-market fit.
  • Create Influence Goals: Strategize how to influence the product roadmap to include features that address community pain-points. In particular, determine who needs to understand the pain-points in order to rally resources to invest in addressing them and then determine the best way to communicate the essential need.

II. Prepare the Conversation

  • Recruit a handful of users  from your community (e.g. spanning geographies, topic areas, etc.) who product team members need to hear from in order to advance their understanding of, and commitment to, addressing the community pain-point. Be mindful to recruit users who represent the diversity of perspectives you want to advocate, including historically marginalized voices.
  • Deeply get to know recruited users via conversations with them about their goals, what is working and what is not working. Listen to and ask questions that elicit the stories that make their pain-points real.
  • Create a run-of-show for an event in which the recruited users share relevant personal stories that will remain in participants’ minds long after the event is over. Make sure the script feels natural to the user spokespersons and give them the chance to edit and internalize the content.

III. Hold the Event

  • Work with product team members to enlist their deep commitment to the success of the event and to jointly prepare relevant questions they can ask the users after they share their stories.
  • Hold the event in which you bring together users with key product stakeholders who need to hear the user stories. Ensure that community members have the chance to share key points via stories. Ask follow up questions to elicit points that you heard in preparation conversations that do not come up on their own.

IV. Follow Up

  • Debrief the event with product teammates without users present to discuss what was heard and what was learned, what options exist for addressing pain-points, what next steps are appropriate, and how the community can be most helpful. 
  • Facilitate ongoing connection between product team members and the community using co-design sessions and regular touchpoints to provide continuous feedback during product design, development, and testing.
  • Celebrate new features that go-to-market that address community pain-points, especially circling back to those who stepped-up to advocate for change by participating in events. This is a huge win for your organization and the community! 

There is nothing like intimate live conversations where stories bring to life the current barriers to user success and the potential future value that can be unlocked with the right product evolution. Check out this conversation with community leaders, external experts and a Meta VP filmed in the metaverse for a fun example! 

NOTES

Be sure to check with company policy and legal professionals to ensure user privacy is maintained and that the process aligns with relevant policies.

Breanna DiGiammarino has 15+ years  experience working with communities at Meta, Indiegogo and the Draper Richards Kaplan Foundation. She uses the process described in her post to evolve product offerings. Reach her on LinkedIn to keep the communities conversation going!

See Also by Breanna:

Get Aligned, Manage to Lead, Sales Matters

Five Steps to Turn a Prospect into a Sale

1 Comment

Developing a systematic approach to cultivating demand for its products and services is a key step in the evolution of every successful organization. Many early-stage leaders long for a silver-bullet solution; that is, they look to hire someone with a lot of contacts and an extroverted personality to hit the market and drum-up demand.  Such efforts usually fail.

Leaders cannot count on building a scalable demand creation system by hiring one super-salesman after another. There are simply not enough to go around. A better strategy is to figure out for themselves how to create demand for their offerings and then hire and train others to follow their lead.

What follows is a sure-fire method to systematically turn prospects into customers that every executive, client manager, product manager, and sales professional can and should add to their tool set.  It takes a lot of work to prepare properly and to execute well in a teaching-mindset, instead of a selling one, but those who are up to the task will be well-rewarded.

Step-1: Describe what you think your prospect is trying to accomplish.

Use all the data about a top prospect you can get your hands on to describe what problem they seeks to solve that your organization can help with.

Arrange a face-to-face meeting with the person in charge of solving the problem, for whom solving it is strategic, and who has the budget and business case to do so. After opening pleasantries, ask the following question in a nice way: “Would you like to know what I think you think is the most important thing you are trying to do right now?”.

You can be sure of a positive response. It is human nature to want to know what someone else thinks you think. At the same time, no one expects what gets said to be 100% correct. They might even chuckle at that thought that you could come close knowing what they think. As a result, your prospect is sure to be interested in hearing what you have to say, even if just for the entertainment value!

This gives you a safe opening to lay out your best articulation of what you think they are trying to do. The beauty of this approach is that to the extent you get it right you gain credibility and, if you get it wrong, you get credit for trying and you will almost always get helpful input to get it right!  If you are right, or reasonably close, continue on to Step-2.

Step-2: Describe what others who have done the same found difficult.

Resist the temptation to sell at this point. Do not talk about how hard or important it is for the prospect to do what they are trying to do.  Doing so will invite resistance and cause the conversation to come to a grinding halt. Instead, talk about others to keep the conversation in a safe space and to invite the prospect to fully engage. Odds are they will lean forward and listen intently because you just might know what you are talking about and say something important.

Now is the time for you to make a good impression with a clear and articulate summary of what you know about the subject. Do not talk about your own organization or your products and services (i.e., resist the urge to start selling) and do not talk about the prospect’s organization or problems. Focus the conversation only on other organizations and what they have struggled with in a way that brings home just how hard it is to do this important thing well and to reveal that you know a great deal about how to do what needs to be done.

Sprinkle specific details about others with whom your prospect is likely to be familiar. Even better is if the examples relate to feared or hated competitors or to organizations the prospect admires and would like to emulate. While it does not matter in general if what you share comes from first-hand experience, from others you know or have worked with, or even from case literature, it is more genuine and adds more to your credibility if it is clear that you have had personal involvement.

In addition to building credibility, the objective of this step is to confirm that your prospect does indeed have the problem you are prepared to solve. If you start by saying:

“Do you have problem X?”

You run the risk that the prospect is reluctant to share the truth.  Instead, say:

“Organization A had problem X”

Thereby creating the opportunity for your prospect to volunteer:

“That’s amazing … we have the same issue!”

The net effect is to build your credibility while drawing out important information for you to use later.

Step-3: Describe how the best have succeeded.

Lay out the approach that the best use to accomplish what the prospect is trying to do. Odds are thatthey will be all ears as you help them see what important things they do not already know, but that they could know if you were on the team. On the other hand, if it turns out that they already know, and are already doing, what the best do then they may not be a good prospect after all.

Here, too, mention how you have personally been involved in some of the “best” cases. Remember that you always have three things to possibly sell:

  • Your company.
  • Your service or product offering.
  • Yourself.

Selling yourself is the easiest and most important of the three and this is your chance to sell yourself and make the sale. Your competence, engaging approach, and evidence of your experience make or break the sale at this point.

Step-4: Describe alternative courses of action.

Given what you know now about your prospect and what others have done, you are now in position to share alternative courses of action that could be followed. There are almost always at least three choices:

  • Continue as if you had never appeared.
  • Try to  follow the best practices you have presented without outside help.
  • Work with a knowledgeable third party to navigate the course you have outlined.

If what the prospect seeks to accomplish is truly important and the stakes are high, it would be foolish to continue as if you had never appeared. If it is hard to go it alone, then the obvious decision should be to get outside help assuming outside help is available and at a price that makes sense relative to the value of accomplishing the objective and the cost of failure.

The prospect could search for others to work with or they could work with you because you are:

  • Present in-person at that very moment.
  • The one who revealed the best path.
  • Brimming with credibility due to the way you made the case.

At this point, you have masterfully created the right time and place to share your approach to addressing the problem with high odds of landing a new customer in the following final step.

Step-5: Recommend next steps.

Use your best judgment to recommend which of the alternatives they should follow. Lay out what the prospect should do, what the prospect should have you do or provide, and what value that leads to for their organization. Make clear that what you would do is an important part of what your organization does and that it would be an honor to turn them from a prospect into a customer and under what terms.

If the prospect says: “no”, to retaining you then it is time to start selling. As they say: “selling begins when you hear the word ‘no’!” While so doing, be sure to learn the basis for resistance so you can factor it into your approach for next time.

On the other hand, if you hear:  “yes”, then you have made a sale by teaching and not by selling. Celebrate briefly and then proceed to package what you have done for future use and train others to so the same.

Example

The graphic below presents key points related to each of the five steps in a real example used by a firm that sold program management and governance services to top government agencies.

See Also

Five Steps to a Sale slide presentation

Three Steps to Selling a Services Work Plan

Whose problem is sales

Prospect to Customer Marketing