"Click five to survive"

[Jean Douglas, CMC]

These four words say it so well.

Background - This is how a colleague explained what they think is happening in their organization's engagement survey. The pressure is on at the executive level to "get the scores up" and is a component of some managers and the executives performance metrics.

My concerns - This is something that I have been concerned about for some time. I have heard in focus groups and other conversations stories of how the intent of the engagement survey and the wonderful information it can provide are being undermined by managers and executives whose focus is soley on the number - the "score".

Some of the things I have heard include:
- Employees being chastised for marking the survey incorrectly.

- Managers when handing out a survey or explaining the survey process telling employees which column to check when answering the survey.

- Employees feeling bad that their manager will lose a bonus if they get a poor mark on the survey (i.e., the engagement score for their unit).

-Employees worried that there is too much at risk to chance answering negatively.

My Question: I was wondering if any one else is seeing this increase in biased answering of engagement surveys?
I also wonder if this is happening (starting to happen) to employees in companies who participate in the "Best Company/Employer" lists?

[Ben Simonton]

Using surveys is part of the problem since any survey or questionnaire can be rigged. Surveys are just more pieces of paperwork to support the top-down bureaucracy that by its nature demotivates and demoralizes employees. It is not surprising that managers and executives try to survive since a top-down run organization will readily throw anyone under the bus and everyone knows it.

The only accurate way to assess engagement is to talk with employees and find out if they are engaged. This is relatively easy to do for an experienced examiner since there is a specific set of rules to follow to create engagement and testing for their existence is not rocket science.

Best regards, Ben
http://www.bensimonton/articles.html

[maggie chicoine]

Thanks for posting this, Jean. I'm anxious to hear the responses. (Backstory: I'm facilitating the review of a 1st survey and making recommendations to top management. Appreciate having a list of "red flags" to guide the snr mgmt group as follow up.)

Maggie

[Jean Douglas, CMC]

Maggie - Could you clarify your role a bit more. I'm not sure if you are making recommendations about the survey process or about the survey results,or something else. I would be pleased to give you a list once I know I am on the right track.

Jean

[Jean Douglas, CMC]

Ben, thanks for commenting.

What you have touched on is another issue that concerns me - the design, analysis and interpretation of surveys by people who do not have the training (credentials) to do so. My expertise is in survey design, analysis, interpretation and advice - I have the degree with the appropriate research design and psychometric statistical analyses courses under my belt (MA in industrial and organizational psychology which focuses on the world of work - not clinical psychology).

Well-designed surveys provide an organization with wonderful information that is useful and eye-opening - the key is "well-designed". That means the goals and objectives of the research (usually the call is for a survey) are well-defined and understood.

The appropriate survey methodology is determined based on the informational needs identified in the investigation into the goals and objectives of the research project. The survey methodology may include using one or a combination of questionnaires, focus groups, interviews, communication audits, etc. While the initial request / thought may be that a survey (i.e. a questionnaire approach) is the correct approach, it may not be the case. Regardless of the methodology used, the same rigour and scientific approach must be used.

It is so easy for anyone to say they are a “survey expert” these days – all they need is access to an online survey tool and Excel. This is one reason we are seeing the quality and use of surveys degrading.

Survey item writing is based in a scientific approach – yes there is some art to it, but a properly designed survey will have the statistical backing it needs – very high levels of reliability (this is achieved through higher level statistical analyses – meaning numbers) and validity (which is also shown through statistics with the exception of face validity which is one component of a validity check).

Survey analysis is another area based on a scientific approach – even though most survey analyses used by organizations are descriptive (meaning x% agreed or y% are full-time employees, for example) but there are other behind-the-scenes analyses – which a good research will use to ensure their data is clean and unbiased. Engagement research usually involves predictive analyses (higher level statistics) to determine what areas in the work place experience can influence engagement allowing the organization to focus on the areas that will give them the biggest positive impact on employee engagement. Again, someone with the appropriate training/education needs to be involved. I need to add an additional warning here – just because someone has a degree (even a PhD) does not mean they are experts in survey design (I have seen some very bad surveys designed by University graduates).

Interpretation is another area of potential abuse. Again, without knowing the appropriate way to look at data it is very easy to jump to the wrong conclusions and in the wrong hands, it is easy to go down the road of finding exactly what you want to find.

I would like to add, that your approach makes a lot of sense in a small organization. However, the cost to interact on a personal level with employees in a large organization is too costly for organizations.

But, I may have misunderstood your post - as to what you suggest.

Please provide more detail about your comment "This is relatively easy to do for an experienced examiner since there is a specific set of rules to follow to create engagement and testing for their existence is not rocket science.".

Jean
dcom.ca

[Ben Simonton]

Thanks for the explanation, Jean. Sounds like you are a verifiable expert in the survey field.

As for my solution, it works effectively for one employee or hundreds and thousands of employees. I personally made it effective in a 1300 person unionized group in New York City. I don't believe that there is a limit as concerns numbers. Cost is negligible.

People are about 4 times more capable than we give them credit, but they need to be treated as if they are valuable before they will become valuable to the organization. If one uses the specific tools designed to treat them as being valuable, engagement will be 100% as employees unleash their full potential of creativity, innovation, productivity, motivation and commitment.

In order to provide a bit more detail of the tool set, here is a a rather simple 10 question test to evaluate whether managerial actions will cause employees to unleash their full potential.

This is a simple test of 10 questions. Rank yourself (or a manager) on a scale of 1 to 10, 10 being the best or almost always, 1 being the worst or almost never. Add up the points for each question.

If you score close to 100, I would expect that your employees will be over 3 times more productive than if your score was 30 or less. In addition, employees will unleash their full potential creativity and innovation, love to come to work and have very high morale. :)

DOES THE MANAGER

-provide regular and frequent opportunities for employees to voice complaints, suggestions and questions, provide reasonable and timely responses, and give employees what they say they need to do a better job? (At least weekly?)

-elicit answers/responses from the team and get them to use their brainpower to solve problems?

-listen to employees with 100% attention without distraction, without trying to figure out a response and with the use of follow-up questions to obtain missing details and suggested fixes?

-refrain from giving orders since by their nature they are demeaning and disrespectful and destroy innovation and commitment?

-treat members better in terms of humility, respect, timely and high quality responses, forthrightness, trust, admission of error, etc than they are expected to treat customers and each other?

-publicly recognize employees for their contributions and high performance and never take credit him/herself?

-openly provide all company info to employees to the extent they need/desire?

-use values and high standards of them in order to explain why certain actions are better than others?

-use smiles and good humor with subordinates, not frowns or a blank face?

-generate in employees a strong sense of ownership?


Best regards, Ben

[Jean Douglas, CMC]

Thanks Ben. I had misunderstood.. I had thought you meant talking to employees on a one-on-one basis.

[Ben Simonton]

Sorry to have misled you. Receiving and responding to complaints, suggestions and questions must be done one-on-one and in group meetings by every level of the chain.

Taking over a group of 1300 employees that was greatly in need of a major turnaround, as their VP I started off using both and thus setting the example for all to follow. The example was of serving employees rather than vice versa as had been done previously in a top-down manner.

After about 6 months of this and some individual training, I required the managers immediately below me to start conducting group meetings (no more than 40 not including having the entire chain of command present). After a few of those, I established a requirement that every employee attend one group meeting each week solely for receiving and responding to their complaints, suggestions and questions. In some groups these meetings were conducted by first line foremen, but most were conducted by middle level managers. All higher level managers were required to monitor these meetings to ensure quality and to help to answer questions.

There were quite a few rules for conducting these meetings such as never shooting from the hip with an answer that was not positively known to reflect the highest standards of all common values.

Hope this explains a bit better, Ben

[Jean Douglas, CMC]

Hi Ben,
I had also thought that you were speaking about a consulting assignment - which was where I was concerned about cost (the high cost of one-on-one interviews with thousands of employees). Thanks for clarifying.

Kudos on the approach. I think you have given us an excellent example of a company whose exec got it right. (that would be you, the VP). However, not all companies have the level of trust of employees at all levels that will make this work.

Your example also illustrated one thing that many companies also miss - "...After about 6 months of this and some individual training..." training for managers and time to make sure it sticks.

I'm interested in knowing if this process continued on and how employee engagement improved over time and business results improvements.

thanks,
Jean

[Ben Simonton]

Jean,

You wrote - "However, not all companies have the level of trust of employees at all levels that will make this work."

How true. But no matter who you are, you must gain their trust. When I took over the 1300, no one trusted anyone and union people generally hated management. In my estimation, they were totally justified as they had been subjected to the traditional top-down command and control approach which by its very nature tends to demotivate and demoralize employees. So in their eyes I was just one more top manager who would very soon mistreat them.

I had to earn their trust and I began by admitting that they had been very poorly supported, that their very poor performance was not their fault, that I had fixed such things before, and what I intended to do about it. The exec who hired me actually told me to disband the group or fix it, my choice. Their customers were disgusted with them. I explained all this to the group when I took over (mouths dropped) and moved on from there.

Productivity after 4 years had risen by about 300% per person by measurement. We stopped measuring but I am quite sure that it continued to rise because that is what self-directed employees do. And I am quite sure that I had converted at least 80% of them to being self-directed from being followers (part of my set of tool). Followers waste a great deal of brainpower following while self-directed people apply 100% of their brainpower to the work 24/7 so being self-directed unleashes a mother lode of capabilities on the work.

I left them after 7 years at the helm.

You also wrote - "Your example also illustrated one thing that many companies also miss - "...After about 6 months of this and some individual training..." training for managers and time to make sure it sticks."

That is part of the secret. I did not give my subordinate managers and supervisors a choice with respect to how they treated employees. I had an anti-robot rule that literally outlawed direct orders except in an emergency or similar situation. They had to conduct meetings and since I and other top managers would talk with employees, it was easy to find a supervisor who was not following the guidance. Besides, everyone had to have an open door policy.

You also wrote - "I'm interested in knowing if this process continued on and how employee engagement improved over time and business results improvements."

I doubt that they changed much after that. My first position at that company was to take over their largest electric generating station, also a disaster in need of a turnaround. Being only 250 people, I did that quite quickly and brought the station to being the cleanest and best operating in 2 1/2 years. Having converted >80% to being self-directed, I learned by watching them over the next 7 years that they never reverted to being followers. In spite of the fact that the three bosses they had in that time ranged from highly technical with little managerial skills to highly political, the station's performance improved in every respect
over the 7 years I was still in the company and privy to the station data.

I would visit them a few times a year and was always met with warm handshakes or hugs and something like "Great to see you, Ben. We are doing better every day and thinking about how we can meet a higher standard tomorrow." So I learned that as I suspected (I have always been one of the 5%) once we become self-directed from being follower we will never revert back. It is just too great a life being self-directed in charge of your own destiny. Unfortunately, IMHO about 95% of us are forced by a very authoritarian society to become followers so it takes some work to get us back to what I consider to be the way our Creator made us.

I am soon to publish a book recounting what I did to get away from top-down and how/why I changed a bit at a time. It spans my entire 30+ years of managing people.

Best regards, Ben

[Ryan Williams, ABC]

Hi Jean,

I hope to see you live at the IABC spring event at the end of the month, but until then good to see you here at the EEN.

As to your original point, this has been something I have been working against for the last few years. Clients are asked to incorporate the survey into a scorecard and then a measure that is not absolute is now being used incorrectly.

A great point is made by Joe Folkman in his book Employee Surveys That Make a Difference, when employees trust the process and believe it is making a difference, the scores may go down. When they engage in the process they can become more critical.

Surveys are tools that need context when interpreting findings.

"Well-designed surveys provide an organization with wonderful information that is useful and eye-opening"
I totally agree but then I may have a bias.

[Susan Stamm]

Hi Jean, I know someone working in multi national corporation ( they are in a US site) and were told by their management team that scores would be used by top leadership to decide which plants would be "kept open" and which would be considered for closing. It is impossible for employees to know if this is a ploy to get good scores or a potential real threat. As you would suspect few were willing to risk it. So much for trusting the data...

Susan Stamm
Author, 42 Rules of Employee Engagement

[Doug Shaw]

Wow this is such a cool discussion. So interesting and relevant. I work in a large corp, upwards of 100k staff. For years we've done an EE survey and for years the perception has been that we've done precious little with the output. When I analysed the last five years results folk are actually killing us on key measures they strongly disagree with and the strongly agree remain stubbornly static. Not surprised, seeing as how the survey almost seems to have become the engagement, if that makes sense. It's rubbish.

The survey had 97 questions with a seven point scale. Hoo boy. Last year we fought long and hard to get the number of questions below 50 (i originally asked for 12) and we now have a 5 point scale. Oh, and we've gone from annual to qtly. Trouble is, we're still doing nothing with the output, it's gold dust far as I'm concerned, honest, open and mostly constructive. Yes, we do seem interested in the scores but honestly even that's fading right now.

I've written to the big boss saying pause the survey for a while and instead, do some meaningful engagement. The survey is contaminated by its past.

So - yes I see the biased answering and have done so for years. I'll try to post some graphs to illustrate my point in a while, may have to hos 'em on the blog, we'll see. Thanks for raising this and thanks to all for contributing such interesting and useful feedback.

Doug