learnability matters

A reputed agro based company recently approached us to conduct an assessment centre for their sales force. An assessment centre is a tool to systematic and objective identification of behaviours in individuals, to support actions such as selection, placement, promotion, career management and training. In an assessment centre, individuals are put through various activities, which help assessors test a set of specific competencies in individuals. These competencies could be to do with the attitude of employees, their skills set, or their domain knowledge. Usually, the set of competencies will encompass all three aspects.

The project would involve identifying competencies for the sales force, designing the assessment centres and undertaking the actual process of assessing the employees. We were quite excited because the project would essentially call for our core competencies in design and assessment.

Contextual Inquiry—what is it and what were our objectives of undertaking the process.
After a close look at the project brief, we felt the need to thoroughly understand the organization and its sales force in particular, before undertaking the task of assessing them. After a round of brain storming, we decided that a contextual inquiry of the sales force would be the best way to understand them better.
A contextual inquiry essentially involves in depth interviews with learners or assesses in this case. The findings of these interviews are meticulously collated and analysed, to arrive at inferences, which could serve as inputs for designing training, or as suggestions for the company, depending on the requirements. In this case, the findings of the contextual inquiry would help us achieve two major goals.

1.Identify key competencies: To design the assessment centre, it was vital for us to clearly identify the competencies that they must have to carry out their existing responsibilities and to move up the value chain.
2.Understand the work environment: To effectively assess employees, it was crucial for us to make relevant activities that would test the identified competencies. And for this, we felt it was necessary to understand the work environment. This included recognising the various stake holders that the sales force contends with every day, classifying the roles played by each rank of the sales force and understanding the various factors that aid and impede their progress.
This would also serve as an important input for designing training at a later stage, after assessing the competencies of each individual.

Protocol Creation
Unlike the next stage of competency assessment, contextual inquiry is not to be seen as a platform to assess employees. While questioning respondents and while collating data, the focus must be to objectively gather and organise information, rather than documenting personal opinions and impressions. To stay focused on this goal, our first step was to create a protocol. A protocol, in this context, refers to a questionnaire that guides the interviewee, enabling us to seek relevant information.

A protocol is usually created by a process of brainstorming, after we have gathered enough information about the company and the work they do. We found it convenient to classify our queries under four broad subheads- demographic data, product knowledge, process knowledge and business acumen. The demographic data would always come first, as it helped respondents warm up to our queries.

We had a different set of questions under each of these subheads for different layers of the sales hierarchy, since each rank played a different role. This apart, we also felt the need to interview the stake holders in the business- the consumers, dealers and the distributors. Protocols were created for these respondent groups as well. In all, we had four primary respondent groups and four secondary respondent groups.

While framing questions, we ensured that each question would collect only one unit of information. For instance, we would avoid a question such as “Where were you working earlier and why did you choose to move to this company?”. This question collects two units of information— 1. details about the employee’s earlier place of work, 2. his reasons for choosing this particular company for his next career move. The problem with such a question is that you tend to lose vital information by clubbing two units of information. For instance, in this case, it may have also been relevant to understand why he chose to jump jobs, an aspect that has got lost in the process of clubbing two seemingly relevant units of data. Also, when we compile data of this nature, we expect to find patterns. For example, in this case, we might want to understand how many employees have had prior experience in the same industry. When several data units are clubbed, it becomes difficult to look for patterns of this kind.

We also took care to frame questions in direct speech. Therefore, a question would read “What are the products he sells”, rather than “What products does he sell”. This is significant because while capturing data it is vital for us to note down responses verbatim. If the question is framed in indirect speech, there is the tendency to also note down the response in indirect speech, which can lead to data loss.

On the field
The contextual inquiry took us to semi rural areas in different parts of the country. Understanding rural markets and business practices was a unique experience for us. An important aspect of our challenge was to try and relate to an FMCG business in a rural setup that was far removed from the very urban, service industry that we work in. We marveled at the manner in which stake holders in a rural business communicate with one another, secure high risk businesses and negotiate their deals. I for one was convinced about the soundness of our decision to undertake a contextual inquiry before we designed the Assessment Centres. It laid open several aspects that we may not have thought about since our reality is far removed from that of that of our respondents.

Following are a list of things we would recommend for an on field research:
1.Follow protocol. And this time I am referring to procedural protocol. This is all the more vital in organizations that are hierarchical, such as our client. It was virtually impossible to seek cooperation from the junior officers, unless they were personally asked to cooperate by the regional or area head. An hour long chat with the person in charge, was all it took for the entire process to be smooth. In fact, several area heads were willing to help us with aspects such as travel and accommodation once they felt included in the process.
2.Decide how much information you want to give each respondent group.For instance, the information we shared with the regional manager was different from what we shared with the sales officers, which was in turn different from what we shared with the distributors and farmers. It is also important that you decide how you would like to introduce yourself prior to an interview. Sometimes, ‘Instructional Designer’ can sound a little fancy.
3.Try and be as natural and impromptu as possible. After the first few interviews, we realised that the paper protocol we held in front of us was not only daunting but also a little distracting to the respondents. Some of them were tempted to look into our sheets to ensure that they were not being marked. Also, it helps to be informal during the interviews as it puts participants at ease. You could even speak in the local language, if the respondent is not comfortable with English.
4.We also realised that the best interviews were those that involved an informal chat over chai, or even in the car while moving from one village to another. We got the least information when we sat in a closed room and interrogated respondents.
5.It is virtually impossible to deal with one respondent at a time. Often, we would have conversations with a distributor and a sales officer at the same time and would end up with information from both respondents. Strictly following the protocol can therefore impede your research at times. Use your discretion, and be alert to the possibility of getting data when you least expect it from an individual.
6.While collecting data, the focus must be on noting a respondent’s answers verbatim, rather than noting down what is correct, or more appropriate. For instance, we asked sales officers about the products they sell. While some called the products by their brand names, others referred to them by their chemical names, another group referred to the products by the local names that have stuck with the products. We noted their responses verbatim, although we were better acquainted with the brand names. While compiling data, we were able to infer as to why sales officers in certain areas called a product by the local name, while others called it by the brand name.
7.Ask for artefacts where ever possible. Artefacts could mean receipts, records, brochures, promotional material, basically any physical object that the respondent uses at work. Questions can often evolve around these artefacts.

The Kern CI travel kit
The first couple of days of the contextual inquiry had me doing the balancing act with my efforts to capture data in various forms. This is the travel kit we recommend!
1.Portable voice recorder: You can almost never note down everything a respondent says, more so in a moving car, or while trying to watch your step in a cattle farm. The smaller and less conspicuous the device, the better are your chances of capturing data without inhibiting your respondent.

2.Note book: A note book is equally necessary, especially when you are conversing with multiple respondents. Make sure your note book had a hard back so you don’t need a hard surface to write on. A medium sized notebook is your best bet. A large book can be cumbersome and lose sheets of paper can be a nightmare to handle.

3.Digital camera: A basic digital camera is great to capture data and experiences. Receipts, records, products and people, there’s always something to be visually captured. The pictures came in handy while making our report for the clients. They underlined the depth of the exercise we undertook, sometimes serving as proof to our findings.

4.Large, single sling bag: A large, single sling bag is the best option to carry your camera and recorder, the protocol sheets, a book and pens. Avoid fancy bags, lap tops and double sling bags that need to be turned over to access the contents.

5.Appropriate shoes and clothing: It’s going to be hot and there will be a lot of walking to be done. Cotton clothing and comfortable shoes are what we recommend. Avoid long flowing clothes and shoes that bite. A bottle of drinking water and some cookies thrown in will also come in handy!

Data Crunching
The final phase of contextual inquiry involved compiling data, interpreting it and presenting the findings in a report. We used excel sheets to compile data, because it allowed us to spot trends amongst respondents belonging to each of our eight profiles. The trends and the interpretation of the trends were compiled into a report that we presented to our client. The report also consisted of suggestions as regards standard operational procedures and the list of competencies that we identified on the basis of our inquiry. Following is the list of competencies we identified for the sales force of the company:
1. Competencies
2. Goal Orientation
3. Planning & Organising
4. Business Overview
5. Customer Relationship
6. Communication
7. Leadership and Team Work
8. Information Management and Analysis
9. Decision Making
10. Sales Management
11. Commercial Acumen
12. Technical Knowledge

Project Bloopers
Every project has them and this one was no different, despite this being our fifth contextual inquiry and the second for a rural project, there were still some pit falls.
Understanding agriculture, poultry and dairy farming: Apparently none of us paid attention to our biology/zoology lessons in school! Our doubts ranged from confusion over the gender of broiler birds to the exact manner of referring to male cattle, to the difference between fodder and grass.

Redundant questions in the protocol: After the first couple of interviews, we realised that some questions about farm care, crop cycles and feeding habits of poultry and cattle were rather redundant. Instead of wasting time asking these questions, we went ahead and modified the protocol. We could have clarified these questions from a technical expert in the field. I recommend undertaking a
pre-research with a sample participant to test out the protocol and modifying it early in the process.

Poor travel kit: Do take a good look at Kern’s CI travel kit especially if you are doing contextual inquiry in the rural areas and are constantly on the move. I made the most critical mistakes. I carried a double sling bag with a heavy laptop in it besides my other equipment and protocol sheets. I had intended to key in data while on the move, but that never really happened. It would take me a good five minutes to get all my equipment together while my colleague who was asking the questions while I captured data would march straight ahead. As a result we missed out on some pieces of data in the initial phases.

Data entry nightmares: We kept procrastinating about data entry while on the contextual inquiry and as a result we spent about three weeks just keying in data after the trip. My below average skills at excel sheets had me committing several blunders before I finally got it right. It’s a good idea to make the time at the end of each day of contextual inquiry to key in data. It also helps to have a working knowledge of excel sheets before you start using it for data entry. After a month of working with data and having a boss who loves excel sheets, I have finally reached some level of mastery with the tool.

So that has been the story so far. The next stage of the project involves designing the assessment centre to test the identified competencies. Watch out for another post to learn more about assessment centres and to find out how we go about designing them.

Share this