“Research – What it really takes to Hunt for Valid Information”
In a formal court video closed-door hearing, an Assistant Prosecutor (AP) is performing a direct examination and challenge of a Forensic Expert Witness (EW), whose expertise is in evidence reconstruction.
The issue is regarding a potential conflict of interest as well as information supplied to the court by the EW. The question at hand is the “credibility” of the information supplied by EW as well as EW’s background. The goal of the AP was to discredit the EW and have him removed from the case.
The AP hones in on a single sheet of paper and starts with his direct examination: “Your testimony, in my opinion, reflects that you are not qualified to testify in this matter. In fact, you are licensed in another state as a medical doctor, specifically a neurosurgeon! You’re not even a Forensics Specialist as you are claiming to be in this courtroom. You live in Santa Monica, California and have never even been involved in a court action. You have no basis of your reports and findings in this case, nor did the court confirm “ANY” of your CV information supplied.
It is obvious that your research and opinion of the results of that research are totally invalid. Isn’t this true?!”
The EW looks at the Defense Counsel (DC) and then the Judge and both the EW and DC start to laugh. The AP is somewhat irritated at the initial reaction and the Judge is intently looking down at both parties with interest. He is not amused at the laughing.
Next to the AP is another party with a laptop jacked in to the internet- the screen turned towards the AP glowing faintly. This party is the AP’s own internal investigator and case hunter. He has a confident, smug look on his face as well.
The EW pauses a moment, looks the AP directly in the eyes, and answers the question with a single, clear word, “No.”
The AP looks at the EW and then there is the famous “pregnant pause”.
“You mean to tell this court that what I stated, in detail is wrong?” queries the AP with a sarcastic voice.
“Yes.”, replies the EW.
The AP is now thinking to himself how easy will be to “take down” this Expert Witness. He has the research, the information (in detail) in his hand with a copy for the court and a copy for the Defense. Slam Dunk.
“I would like to have this “Summary” of the EW marked as State’s Exhibit “23” says the AP confidently. The AP hands the copies to the bailiff, who then hands one to the Judge and one the Defense counsel.
“No further questions.” States the AP, who then sits down smiling.
The Defense counsel (DC) looks at the document, and then shows it to the EW.
The DC rises and asks the EW, “Is this information accurate?”
The EW replies, “Yes.”
“Are you sure?” asks the DC. The AP has a puzzled look on his face.
Again, the EW replies, “Yes.”
“Is this person you?” asks the DC.
“No.”, replies the EW, who is now looking at the AP.
“Your Honor, this individual, reflected in Exhibit 23, is true and correct, but it is NOT my EW. It is a person with the same name, but definitely not a Forensic Expert Witness. Further your Honor, this “other” person lives in San Fernando Valley. I would like to submit the CV of this individual which clearly reflects that the AP’s information is highly inaccurate. I would like to have this marked as Exhibit 24.”
The AP stares at the copy of the exhibit handed to him by the bailiff. The Judge looks at his copy, then looks at the EW and then turns his attention to the AP.
“It appears your due-diligence is more than just flawed counselor. You have wasted the court’s time, MY time, the Defense counsels time and the EW’s time based upon this?! What did you use? Google?! barks the Judge. It appears that the “credibility of your case and research should be in question, which it IS!”
Yes, the Assistant Prosecutor has just been blown out of the water and his case has now been severely tainted. Moreover, this is a true account of a criminal case that took place in the State of California. It isn’t the first time that evidence, based upon poorly-researched and validated information has resulted in disaster.
Oh, the Slam Dunk? The case was dismissed with prejudice.
Due diligence IS research.
Validating information is PURE research.
It appears that the process used for much of what we need to investigate and verify (validate) is gone so electronic, and there is so much information available, that the process of research not become easier, but actually much more difficult.
There are wikipedias (e.g. wikis) of all types; however the original Wikipedia was formed as an open-license resource (http://www.wikipedia.org/) that supplies a plethora of information from a public-authoring system. From businesses and politics to military, UFOs and gaming. But that information is not always accurate. As a matter of point, the information within many of the on-line resources is skewed, incomplete, tainted or just dead-wrong.
There are thousands upon thousands of resource site of all types that it boggles the mind. Associations that have their own “interpretation” of things. Their own “definition” of things. Organizations that have literally re-written history to their liking and how they put their spin on things.
Yet, the internet is the number-one research source for information gathering. You are aware that there is absolute proof the world is flat; that the sun circles around the earth; that eating ground Quartz (specifically Amethyst) cures Cancer; and that no one landed on the moon and it was all prefabricated in a studio. That’s what the resources tell us so it “has” to be true, right?
Almost all legal books, especially State Statutes and of course the United States Code (USC) have gone to DVD. Yes, the paper books (and monthly supplements) are still available, but the DVD versions are so much cheaper.
However, did you also know that those supplements, where laws have been amended, repealed or added are on “additional” DVDs that are submitted monthly? The hard paper supplements are added into the binders or books, but the DVD supplementals have to be referenced directly. This means researching the research materials twice (or at least until a whole new version of the updated code is produced on DVD).
That little, tiny footnote at the bottom of the written paper supplement references that a specific issue of law, critical to a case, has been repealed. If it was missed by the attorney or researcher, the entire premise of their case, whether it be the defense or the prosecution, is “at risk” of being compromised.
Juries love to see mistakes made like that.
In business, research and validation is a matter of accuracy and collaboration.
Being able to produce clear proof of the facts and supporting evidence (e.g. information) that redundantly shows that the “basis” of the research is correct and “best effort” has been applied to what is being represented as fact.
When supplying marketing intelligence, the research is crucial in the case of accuracy. A companies’ entire marketing strategy and manufacturing could spell financial disaster if that marketing intelligence was wrong. That the research and validation of the supporting information was inaccurate. It could cost millions, or even destroy the company financially.
In military affairs coupled with the political harness, research is the keystone to “every” decision made. Those decisions must be made on accurate and validated research.
If a mistake is made, the repercussions could be disastrous. I think there is not a single reader out there who would disagree even if they are not history-buffs.
I could give you thousands of examples, but let us not digress from what the subject is about. The accuracy of research and validation of facts is not just an Internet methodology.
The Internet is NOT the “all-knowing” wellspring of truth. Far from it actually.
The term, “Research” is probably one of the most abused words used in ALL professions and disciplines. Validation sounds more like something you do with a parking lot ticket when you come to collect your car after an appointment. But the reality is that research is a science and a skill. In my own opinion, there should be a Master’s Degree (at a minimum) in the discipline of “researching“.
Research is not “contingent” upon ONE source of information.
As a matter of point, a good researcher will play “devil’s advocate” with the work accumulated. Challenge it. Attempt to find holes in it. But the “better” researcher will seek out numerous sources of the information needed to be able to collaborate the targeted subject or issue at hand.
It is a matter of (1) constantly acquiring numerous sources, evidence and materials; (2) compiling and analyzing that information acquired and the “sources” of where it came from, how it was acquired and by what source as well; (3) making a determination that does NOT include validation until it is challenged and sustains the challenge by the person doing that research; (4) utilizing other individuals and assets that can supply, collaborate, reinforce or “successfully challenge” the determination made by the researcher; (5) if there is a successful challenge, it means going back to the drawing board; and finally (6), if there is collaboration of the original determination as a result of other input/review/research by other individuals and assets, then and only then can it be validated. Keep in mind this does not “guarantee” that the research IS 100% correct or accurate, but it would assure best, reasonable effort has been used.
As an example, with the development of the rocket, there were numerous failures (some of them quite spectacular), but that was not due to “flawed” research. It was due a “new” field, untested where best effort was used in “development” of a successful rocket was the goal. Research, in this discipline, runs in a totally different manner, but the assembly of facts, tests, theories and such are still treated the same.
When it comes to Anti-Money Laundering (AML), or Virtual Currency Frauds (VCF) and the ways to detect them, track them and identify those who are committing the crimes, “accurate” and up-to-date research is critical to the success or horrible failures that would occur.
Anti-terrorism is a big buzz-word more than ever now, and it can mean way-too-many things to people. In fact, let us make a small change to the words and call it Anti-Terrorism Research (ATR).
As a crystal has numerous facets, they still are all are connected to the same gemstone.
So as it should be with ATR.
ATR would address multiple disciplines from domestic terrorism to maritime terrorism.
It would address the methodologies and strategies used to go head-to-head with the “hardest” adversary(s) we have ever faced.
It is not the bullet, bombs or drone attacks that will be the successful linchpin. Nor will applied sanctions that hopefully will apply the necessary pressure to change the policy/position of non-cooperation of a country.
Yes those are essential components to fighting or taking proactive steps against terrorism (as we understand it), but it is the RESEARCH that is the critical keystone that everything is based upon. Such research must be redundantly accurate, and what makes that so much harder is that it is comprised of many facets that must work within a common denominator.
Whether it be companies or government agencies, private enterprises or the military, the “transparency” of information that is accurate and SHARED properly, especially between countries, must be researched AND validated, regardless of the source.
As an example, you would never depend upon “one” source of information on terrorist methodologies of new terrorist cell groups. Not just one source.
Jumping to businesses, if a merger is being contemplated, you just don’t depend upon sales reports and news blogs. Obviously you dig into the books, the financial records and many, many other area (facets) to properly research and determine accuracy, honesty, compliance and finally if it is represented “Correctly”.
With Homeland Security, it is a massively-large gemstone with many, many facets.
It is totally dependent upon research results when it make decisions. Yet, many of the decisions are made independently even though there is an overall policy that is to be followed.
It is a paradox.
Research is comprised of the components and methodology which is universal in ALL disciplines. It MUST contain the “core” components for it to be valid and accurate under the “Best, Reasonable Effort (BRE)” rule.
From military and government agencies, to public and private business entities, research is the heart that runs their operations. Without it, they are literally quite blind if not brain-dead – but the body keeps functioning (sort of).
The mechanisms of research is been severely overlooked and somewhat ignored.
Why? Because it is assumed that people “already know” how to research.
This assumption is wrong – dangerously wrong.
Educational instructions teach courses on numerous subjects. In the sciences it is a matter of facts, premises and known laws of science.
In the business realm, from marketing to production, it is a matter of learning policies, systems and procedures, and strategies handed down by C-Management.
In government it is a matter of political members and committees that develop laws, policies and strategies to be used domestically and internationally.
With the military, it is a matter of history, tactics, strategy, logistics, working with what is immediately available, learning to improvise, to acquire intelligence that is accurate and finally follow the guidelines/limitations handed down by the government.
All of these have that common denominator, in that research IS the keystone to every single portion of their operations. From the simplest to the most complex.
Research is designed as a living “advisor” to depend upon that is comprised of one or more people that have performed detailed, thorough and “reliable” investigation (e.g. looking into) and validation resulting in an opinion, a policy, an answer, a solution or a direction to take.
However, the word “reliable” is also a keystone (a “facet of a facet”) since thousands of pages of documentation that comes from various sources that clearly conveys that 2+2=5 is obviously wrong. Yet, with this analogy, we will improperly accept that answer as “accurate” if there is collaborating information and it comes from numerous resources. Without validation, it makes no difference of the “amount” of supporting data that is available and used – it is still incorrect.
Research also contains a combination of gut-feel, hopefully “non-biasness” (which is the hardest one to perform consistently since we are all biased regarding one matter or another), and most important, “honesty”. The issue of honesty incorporates situations where the researcher comes to an impasse, or inability to acquire needed information, or just doesn’t know how to approach the task of researching the specific subject at hand. Adding to the list of reasons there is a final one, and that is the results found are not what was expected. There are cases where research is requested to “prove” a position, policy or expectation and in many cases the results are contradictory, or not acceptable.
One of the great enemies of honest, non-biased research that has been properly validated is “rationalization”. If one looks up the word “rationalization”, it can mean a number of things but one definition is clear, and that is “coming up with an explanation, reason or “excuse” that isn’t true”.
Rationalization does NOT belong in research and especially not in the Validation process. Worse yet, are those who requested the research and validation to be performed on a specific project or assignment, and after reviewing all the results, “overrides” the results of the research by supporting their position or decision by “rationalizing it”.
There is another group of factors the effects both the “quality” and “accuracy” of research:
- The inability that the research task is “over the head” of the assigned person(s)
- The inability to ask for help (e.g. fear of being considered underqualified to the task or would be viewed as incompetent to the person’s superiors and peers)
- The fear of failure
- Prejudice, Favoritism, Preconception (Biasness)
- Preconceived Notion or Conclusion
- The fear of the research being wrong
- The lack of understanding “how” to research the topic or task at hand/assigned
- The amount of time given to perform a valid research
- The person doesn’t know how to or where to acquire the information/data to provide a basis for the research and validation
- The person doesn’t understand the research materials
- The list goes on and on…
Again, it goes back to HOW research is taught. Honesty it is not taught since it “assumed” to be in the equation of the process of performing research and validation.
It is an assumed talent or ability of a person. Yes, in the workplace, military or government, there is the matter of going to various other personnel and “bouncing” requests and ideas off of each other like a handball against the backboard, but that is like the proverbial “street corner advice center”. Is that honest research?
Hwang Woo Suk is not a name remembered, or even known by many, but at one time he was a hero in the scientific community with his “breakthroughs” is stem cell research and cloning procedures/results. His work was “researched” by a “specifically selected group” that performed the due-diligence and validation of “his” research and results.
However, when independent “honest and non-biased” researchers performed the same due-diligence and validation process of Hawang’s work, it was found to be totally faked.
In simple terms, Hwang was a fraudster. He finally admitted falsifying much of his work. Later, embezzlement charges surfaced and he was disgraced from the scientific community and ended up with serious legal and criminal problems.
Mr. Suk’s problems are not the only issue that, as a researcher, one should be concerned with. There was no mention of those who “authenticated” his work. Those who performed the “scientific” due-diligence (research and validation) of his work and results. Those who “profited” from his findings. Yes, honesty is a key element (a core component) of research.
Research is a “discipline” and it has a “core” group of procedures that always will be in common with any project that requires research and validation.
In our institution, we teach and train interns in the discipline of researching, and then in the specialized disciplines that research would be required. This is something that is NOT learned in a week or two. It takes years of commitment and hard, focused work to develop one’s ability to be a “good” researcher.
But to be an “excellent” researcher, one has to be able to:
- Develop their sources of information;
- Dependable and accurate information;
- Redundant but independent sources of information;
- Information sources that can be validated as accurate and authentic;
- The time frame of the information which could be “snap-shot accurate” at that specific time and date, and become inaccurate in a matter of minutes, hours or days and be able to identify such discrepancies if found;
- To be able to honestly assess the research data collected and assemble it into a timeline or structure that makes sense and can be defended WHEN challenged (which it should be);
- That additional information may be required to be added to it, amended or deleted where it may necessitate the entire work and assessment to be redone;
- To know one’s limitations where help is needed and ASKING for it;
- The ability to research the materials point-of-origin (e.g. author, producer of the information and their background);
- To validate the credibility of the source of information used in the research performed;
- To NOT use questionable, but good-looking facts and statistics to reinforce findings;
- To never use hearsay data UNLESS it is the only information available and it is stated clearly that the reliability of such information should be seriously considered as hearsay and nothing more;
- To learn NEW ways to acquire data and information that use standardized and accepted protocols for collecting research data; and
- To develop a premise(s) and proof it/them to where the challenge can be successful or not.
The steps to producing the end-product of research and validation can be broken down into a “brief” number of basic of steps:
- First, to be able to assemble all the collected information, data and intelligence and assemble it in an outline cursory format;
- Second, then produce the collected data, information and intelligence in an expanded format which as much supporting detail possible;
- Third, be able to render an opinion or confirmation (e.g. yes or no, or be able to make a definitive statement of facts, conditions or position with the required recommendations (if requested); and
- Forth, be able to produce a clear and concise summary page that is at the BEGINNING of the work product produced that clearly gives the answers or information required that was the basis of the required research – the rest of all of the information, intelligence and research materials are “supportive documentation” that would be footnoted or properly referenced within the report/document/findings.
Mind you, let there be no illusion that the above structure is just one facet (simply addressed and not going into extreme detail) in the methodology of researching procedures.
There are numerous variations that dictate changes and additions depending upon the application(s) of research being performed at hand, such as the risks or threats involved, the designated “importance” of the matter or issue requiring the research and the time frame given to perform the research and validation.
I have participated in “Think Tank” projects and their environments, some that have lasted a few days, some a few months and then those that are literally on-going for years (especially when addressing national/international policies and actions).
The famous “what if” scenarios warrant the greatest challenges to researching and those who perform the research in the Think Tank environment. The demands are great and require a collective unit of highly skilled and seasoned professionals; however that does not mean that the research gleaned will flow cohesively.
As a matter of point, it is like having several individuals, all working on the same project or assignment, who come up with several different facets of information that needs to be combined into the single, cohesive research project with agreed upon results or findings.
This is a matter of finding a common denominator for ALL of that data, and in many cases THAT is what takes the greatest amount of time. No single person in a Think Tank wants their work “modified” to fit a common, single research project. Yet, the objective is to FIND that common denominator and integrate it as a POLICY to be practiced by all those involved from the “start”.
Validation is the act of proofing the research. It is not commonly discussed since it usually performed AFTER the research has been submitted by those reviewing it.
This is a disaster in the making. Validation is “another” keystone in successful research. Without it being performed correctly, it opens the door to research that ultimately will be found out to be wrong, incorrect, inaccurate or dangerously misleading. In international policies and politics, this is a deadly game if played. Validation is a stop-gap and should be considered nothing less than a device to “proof” the work that was used to produce the results of the research as well as the body of the research. It requires even more skills and abilities that a general researcher would normally be able to apply or even understand.
Now, coming back to the initial resource I initially pointed out, the internet, no matter how vast, nor matter how large is NOT to keystone to research. Yes, it does have its place, but for the most valid, most accurate and most honest sources of information, it will NOT be found on/in the net. In some cases it may “point” to resources which are not found on the internet which are truly reliable.
A good researcher develops a reasonable number of outside resources that can be trusted. An excellent researcher perpetually hunts for current sources of information, from people and entities that can be validated as credible, reliable, and most important will be able to simultaneously develop their “validation” skills to the highest degree possible – and not stop learning.
A good researcher can be an excellent teacher in the methodologies of research; however an excellent researcher will “always” be a full-time student, constantly seeking new (or old) methods and resources to experiment with, constantly seek solid sources of information or methods of acquiring solid sources of information, and have an on-going development of their validation skills. This includes constant self-assessment of their methods of performing research and validation as well.
Looking at what most of us understand what the word “research” means it includes, but is not limited to:
- Look into
Looking at one of those definitions, we see “study” as a clear definition of research.
To us, study (e.g. studying) would be a starting point for those in school learning a new subject. In many cases, “research” requires learning about the target subject or task from the beginning. A person performing the research may not have a clue of the subject matter or target, yet the very first function would be to “Study” it.
Therefore, “learning” would also be part of researching and a critical component of it as well, since learning about it improperly, wrong or skewed would be building the entire research project on an invalid premise.
Foreign policy is built on pure research. Yet, the information needed for the research data may be lacking, incorrect or incomplete. This means that a variety of tools need to be used to acquire the most accurate information possible. When you think about it, a “Spy” or “Operative“, by definition is parallel in a majority of ways to a researcher since a “Spy” IS a researcher in the purest sense. Their objectives are the same. It is just a matter of a different form of acquiring the data being researched.
When you look at military Drone Strikes, the amount of “research” that goes into the planned target is massive. It requires several facets of research that is “validated” to the best ability of those assigned to perform the task (the research).
From Ariel photographs, to intelligence gleaned from ground contacts. From people in towns to spies. The number of potential data points for research information is massive; however the “reliability” of such intelligence is a different story. It will be the “validation” of the final report comprised of all the research performed that will lay the groundwork for that Drone strike.
In this case, as with many other facets, there are “variables”. Some of them are rock-solid (by definition), others are fluid and constantly changing. When one thinks of a target of opportunity, one of the key questions is “what about collateral damage”? The civilians? What this proves is even after the research has been performed, variables that surface “force” the issue of reconsidering options. This in turn, means MORE research, and that too must be validated.
Homeland Security constantly is dealing with intelligence reports that are either (a) leads turned in from other resources, or (b) self-generated on-going research. In either case, the lions-share of the United States public will never know just how many terrorist plots were foiled because of vigilant research. This, in my opinion, is not only taken for granted, but is, unfortunately, the way it has to be. Disclosure of foiled attempts would compromise resources that are used for intelligence, which, in turn, compromise and terminate key components used by the research system.
Researching, in its purest state is “investigation” of events, facts and information that represents past, present and projected future data (based upon intelligence and other facets of the research performed).
Instead of acting “reactively” with how we approach information gathering and intelligence within ANY aspect of what we deal with, whether it be public, private, governmental or military, the need for researchers is seriously needed to address research “proactively”; however the methodology and educating in the ability /methodology of researching is paramount and should come first.
We plan to follow this initial article on Researching over the next several months with on-going strategies, methodologies, systems and procedures for the various facets of how you identify, create and tune your Researching Abilities, Strategies and Skills.
Yes, that means we will be looking at numerous industries, professions, the government sector and its agencies, Homeland Security (since it is not only large, but exceptionally complex), law enforcement and military sectors, security-in-general (e.g. Maritime, Utilities, Water, Fuel, etc.) as well as international applications.
We will also address what “Validation” actually means in relation to researching.
We consider validation to be the “auditing point” of all research that is performed.
Validation can be even more complex than the research it is reviewing/analyzing, but discussion on this will be saved for a later time.
We have been teaching and training Research Methodologies with the accompanying Strategies, Systems and Procedures since we provide research and development services literally on a full-time, on-call basis. It is our “primary discipline” (e.g. doctrine) that is one of the most important facets of our corporate jewel that makes RSI who we are.
It is truly one of the most important “core components” that represents our history, credibility and reputation, nationally and internationally for over three decades.
Dr. Mark D. Lurie,
CEO, Threat & Fraud Assessment (United States)
®Google is a Registered Trademark of Google, Inc.
BRE is credited to one of the finest, most recognized teachers, Mitchell Lurie