In the following, the Commission will summarily point at certain other errors and defects which together and alone entail that the credibility of the three stated articles and PhD dissertation is considerably reduced.
A weakness in this study is that the researcher in charge (i.e. Jon Sudbø) had full access to all patient data. This is a weakness because the study according to the articles and dissertation was to be a blinded historic prospective study, i.e. a study in which by going back in time one may follow patients’ future development. Blinded means that those who made the ploidy analyses and dysplasia classification were not to know whether the patient subsequently got cancer or not. In this case whether patients who had first got the diagnosis white patches in the oral cavity (dysplasia) at a later point in time developed cancer. Such a transformation and its frequency would be able to say something about the extent to which the ploidy in cells from white patches could be used as a sign (a predicative indicator) of future oral cancer. But because the study was an historic prospective one, one in fact had the answer on one’s hands, by knowing how many patients developed cancer. Therefore, the person who was in charge of the research itself, Jon Sudbø, should not have had access to the patient information before the classification of dysplasia and ploidy had been completed, what both he and Reith according to Sudbø himself had.
As previously mentioned, the Commission will at this point state that in most observation studies a “blinded” classification can be and is performed, even if those who perform the classification could have cheated by opening the blinding. Under normal circumstances, one has sufficient trust in the researchers who carry out the study so that the requirements as to blinding are not as stringent as in randomized studies, for example.
However, the Commission will point out that a lack of blinding as regards Sudbø, may contribute to explaining how he was able to manipulate for example the ploidy analysis in order to show positive research results, for example by including patients who had had oral cancer (and who for that reason should have been excluded). It should be noted here that another, alternative, explanation is that the analysis results have been manipulated later on.
New England Journal of Medicine 2001 page 1271 reads:
”all 150 patients had been … enrolled in a follow up-program, which, through an updated national register,
had hospital-based access to the place of residency of Norwegian citizens. No upper limit was set for the
duration of follow-up. Patients who were given a diagnosis of dysplasia were scheduled to have an annual
examination, which included inspection of the oropharyngeal mucosa and palpation of cervical lymph
nodes. Biopsies were performed at these follow-up visits if previously unrecognized white patches were
detected, white patches recurred after excision, or previously recognized patches had increased in size. No
patients were lost during follow-up, although data on seven patients who died of unrelated causes were
censored at the time of death.”
There was no fixed follow-up program for persons with a diagnosis of dysplasia and who did not have cancer at this time. Admittedly, some patients were probably summoned to a check, but this was not done systematically, and it was not necessarily a question of an annual follow-up within the framework of a “program” either, as is the clear impression the article gives. It is also a fact that such a follow-up did not comprise all patients. However, Sudbø maintains that the program did not form part of his scientific project, but that at this time the routine at both Haukeland and Clinic for Oral Surgery and Oral Medicine was to set up a follow-up agreement a year later if a biopsy had been performed. Since this routine was institutionalized, Sudbø believes it is justified to call it a “program”.
However, the Commission finds this argumentation doubtful. The Commission furthermore is very much in doubt that no patients dropped out of the follow-up program.
The Cancer Registry has also difficulties in understanding that which is described in the article, and clearly expresses that this cannot be correct, as this is not the way things function in reality. On the other hand, obviously no others, for example coauthors, reacted to this description.
On this basis, the Commission finds that the way this has been described in the article in the best case is misleading.
Errors as regards smoking and alcohol habits
Table 1 and the text in New England Journal of Medicine 2001 refer to smoking and alcohol habits and a follow-up program for dysplasia patients. New England Journal of Medicine 2001 states on page 1272:
“Patients with confirmed use of tobacco or alcohol were given standard oral and written information on risk
factors for oral cancer, and this information was repeated at each follow-up visit. Data on tobacco use were
reconstructed from the medical records or by the use of telephone interviews, in which the patients were
asked about their use of tobacco at the time of the initial diagnosis of oral leukoplakia (no history of
tobacco use, former use of tobacco, or use of tobacco at the time of the initial diagnosis.”
The Commission has discussed this information with specialists with knowledge of this. It should be noted here that the Cancer Registry has had access to and has reviewed all the referring letters.
The material from Gade comes from hospitals all over Western Norway, from Møre og Romsdal in the North to Rogaland in the South. There is no information that the clinics who sent biopsies to Gade’s Institute participated in any form of scientific study which concerned smoking and alcohol habits or the like. Detailed information on smoking habits is therefore seldom stated on the pathologists’ referring letters, a fact confirmed by the Cancer Registry’s review.
As did these experts, the Commission believes that the information that all patients who smoked or used alcohol were advised, either orally or in writing, of the hazards they exposed themselves to, is erroneous. It may happen that clinicians – to a larger or smaller degree – informed their patients of such facts. Sudbø or other persons involved in the research project have clearly enough not been providing such type of advice, and it is impossible that they can have had access to this information, and they also never had any personal contact with the patients.
In New England of Medicine 2001 it is alleged on page 1274 that the information about smoking habits was achieved from the journals of 100 patients. A further 37 patients were telephoned and gave such information to Sudbø et al. There is nothing to indicate that such information can have been retrieved from the patient journals as alleged. There are no grounds to believe that Jon Sudbø, for example, has had access to the medical journals of these patients. Copies of the journal would in such a case have had to be sent to Sudbø, or he may have traveled across Western Norway and retrieved information that way. In any case, medical journals will often lack systematic information on smoking (not a smoker, present smoker, previous smoker). Correspondingly, there is little reason to believe that the information on alcohol use has been accessible in any systematic way.
Sudbø states that he himself, together with Puntervold and Reith, called around to the 37 patients for whom smoking data allegedly were missing, to supplement the data basis. However, no log for the carrying out of such telephone interviews exists. Sudbø has explained that information obtained in such telephone conversations was noted down on loose bits of paper and/or plotted directly into the data file. Puntervold and Reith, however, state that they have not called any patients about this. Moreover, the Commission finds it improbable that one called to a sufficient number of patients about whom there must have been a need for supplementary information. It should also be noted that 48 persons were dead in 1995, that is before the time
information allegedly was retrieved (see table in Annex 4). In other words, the Commission does not trust Sudbø’s explanation on this point.
In this context, it must be remarked that Sudbø as the main supervisor contributed with data on smoking to one of his research fellows in 2004/2005. However, the fellow has this year, right after the submission of the PhD assertion, based on this case reevaluated the data. The fellow found that there is no conformity between the data on smoking the fellow received, and the data on smoking stated in New England Journal of Medicine 2001, although the raw material is the same. The fellow found, i.a., that all the patients in the material received from Sudbø are smoking, whereas in New England Journal of Medicine 2001 there are patients (27/150) who have not had any consumption of tobacco. The fellow’s dissertation has for this reason, among others, been retained, and will probably be retracted.
The data used in New England Journal of Medicine table 1 is incompatible with data one may reasonably expect that there was access to at that time. The Commission is in strong doubt that Sudbø has had access to complete sets of data about smoking habits like he has given the impression of in i.a. New England Journal of Medicine 2001 and to his research fellow. The information on smoking and alcohol habits appears to the Commission to be partly fabricated.
In this context, the Commission finds reason to note that, in spite of flawed data, New England Journal of Medicine 2001 page 1277 states that the information on smoking was taken into account in the multivariate analysis, but that it did not influence the results. At the same time, it is stated that reliable data for alcohol use were not available for more than half the patients, and that the use of alcohol was therefore not included in the analysis. Both these points have been accounted for in an elegant and convincing manner, and reinforce the reader’s belief in this researcher, partly because the researcher appears as honest and thorough, and partly because he points out possible weak parts of his own study. This elegance in the presentation of research results may be a possible explanation of why no one found any particular reason to question and critically check the publications, more than what was done at some times.
The Commission has put the question of whether the two articles in New England Journal of Medicine 2001 and J Pathol 2001 have so many similar features that they must be considered as a double publication of the same research results in contravention of good research practice. The main results in New England Journal of Medicine 2001 and J Pathol 2001 are coinciding, but nevertheless such that the J Pathol article provides more details as regards the four pathologists’ dysplasia classification.
The Commission has submitted the question of a double publication to Reith and Sudbø. Reith acknowledges that it is a defect that there is no cross reference. However, he refers to the article in J Pathol 2001 to a much larger degree than New England Journal of Medicine 2001 discussing in-depth methodological and conceptual matters, of particular interest to pathologists. Thus Reith believes that the article has its own value and that it was not possible to include these matters in the article which was to be published in New England Journal of Medicine. Sudbø refers to the subject double publication being a relevant problem at the time of publication, which was discussed, but they concluded that it was within what was acceptable. Reith has also referred to the lack of cross reference having its explanation in both manuscripts being submitted simultaneously, and that New England Journal of Medicine does not allow cross references to articles which are submitted only. A cross reference was according to Reith regrettably forgotten at the later time when the manuscripts following several rounds finally were accepted and ready for publication.
The Commission is in doubt regarding this point, but has concluded that there was no obvious double publication. The Commission nevertheless chose to comment on this point to show that the question has been considered.
Confusion or manipulation of pictures?
Media has paid lots of attention to the fact that the same picture in New England Journal of Medicine 2001 appears twice, but is stated to represent two different patients. This is the basis for the expression of concern that the editors of the journal have published. It is a fact that it is the same picture, but in different sizes. This is an obvious error. Jon Sudbø has admitted this, but alleges that it was due to an excusable confusion.
The Commission has not considered this to be an important point and for that reason not pursued the matter further.
To make a thorough and checkable investigation of the actual facts has been a very difficult task to perform. This is partly because the research is advanced and based on a large amount of data which can only be understood with knowledge of technicalities linked to specialized patient studies.
Another reason that this type of investigation is very difficult is the lack of precise documentation of all steps in the research process. This problem is important and not unusual, and not unique to this case. Furthermore, sets of data and lists will exist in several versions with different names and it may be difficult to know which changes have been done, by whom and when. The investigation is further made difficult by the fact that for reasons of personal data protection and secrecy one cannot use person-unique identification of the
information, but must make use of sample numbers, preparation numbers or block numbers, and many data files are entirely without identification. In this section, the Commission will try to summarize section 4.2.7. As stated, the Commission has concentrated its investigation on the article in New England Journal of Medicine 2001, because it is definitely the most important publication in Jon Sudbø’s research career. It forms the basis for several contemporary and subsequent original articles, i.a. a similar article published in the same journal in 2004 (New England Journal of Medicine 2004).
The main analysis and the main findings both in the 2001 and 2004 articles in New England Journal of Medicine are based on the same patient material consisting of 150 patients. Samples from these 150 patients have i.a. been classified according to DNA ploidy in samples from dysplasias in the oral cavity. This ploidy classification was in the 2001 article shown to be a strong predictor of future cancer development. This finding was reinforced in the 2004 article with a further follow-up and 11 new cancer cases.
The Commission has received the data file (Sudbø8) from MD Anderson, with confirmation that it forms the basis for the New England Journal of Medicine 2004 article by reproducing Figure 3B among other things.
This file agrees with a list the Commission has received from Reith. This list again agrees with a list (L29) which the Commission has received from Head of section Danielsen and which is produced in 1998.
Having made a thorough evaluation based on the Commission’s own review of a large number of lists and data files (Annex 3: Files_lists.doc), and the Cancer Registry’s extensive retrieval of all referring letters with relevant dysplasia diagnoses, and the reconstruction of Sudbø’s data material with the linkage that the Cancer Registry made in 1996 with a file of 226 persons (63 of 226 patients who came from Oslo were not connected at the time because they lacked personal identification numbers) as the starting point, the Commission has found the following fundamental problems with the central patient material of 150 patients used in i.a. New England Journal of Medicine 2001 and 2004 articles, and in a series of other articles:
The 150 patients do not exist, in the sense that as far as the Commission can see, it is a matter of a maximum of 141 patients. The reason for this is that some patients are represented by several samples which in the aggregate give the number 150. Letting a patient reappear with several dysplasia samples is contrary to the description in the articles, and does not give any scientific meaning in this context. These replicas of persons are therefore invalid, and should have been excluded from the file. The Commission has not been able to determine the precise number of replicas (this is because 8 of the 150 lack block numbers on the Reith list),
but has concluded that the number of different persons in the file is at the most 141.
Failing exclusion due to simultaneous or previous cancer
Among the patients who the Commission has been able to identify in the Sudbø8 file there is a large number of patients who should have been excluded because they had had oral cancer prior to or simultaneously with the dysplasia in question. The Commission has found that at least 69 persons cannot be included, as they already had oral cancer at the time of the referring letter.
The Commission has only been able to retrieve ploidy analyses for 69 of the observations (65 different persons) in the Sudbø8 file. This is based on lists of ploidy analyses which Puntervold has obtained from the Sudbø material (Gade/the Odontology). A far greater number of ploidy analyses (>150) have been performed, but the same patients appear several times.
Age distribution is not correct
The age distribution in the original material from Gade and the Odontology does not correspond to the age distribution in the material which formed the basis for the New England Journal of Medicine 2004.
Out of these four fundamental problems, it is number 2 which appears as the most serious, and which the Commission with a high degree of certainty can determine, because it is underpinned by independent information from several sources. When more than half of the central patient material is excluded, also all the results in both of the New England Journal of Medicine articles fall to pieces as well as all further research based on these.
The Commission finds that Jon Sudbø has not had access to the number of patients which he states to have had, including that the dysplasia classification and inclusion process have not been made in an honest way such as described. The Commission in particular refers to the fact that out of the 150 of allegedly 263 patients included in the study, more than half of the included patients should have been excluded due to a preceding or simultaneous diagnosis of oral cancer.
It is thus evident that both articles in New England Journal of Medicine and all further research based on the same material cannot be correct or be based on reality, apart from being based on qualified guesswork. Publications based on this raw material must for that reason be retracted, see furthermore section 4.4.
There can be two explanations for the errors discovered:
An unfortunate combination of excusable errors and misunderstandings, as well as failing competence, alertness and thoroughness.
Scientific dishonesty, i.e. the fabrication and manipulation of research data and consciously misleading research reporting.
The Commission finds that the errors and defects discovered are too many, too large and too obvious to be ascribed to excusable errors, incompetence or the like. The Commission finds that data have been manipulated and fabricated, and probably adapted to the findings one wanted to arrive at.
The Commission finds that Jon Sudbø has been alone in the manipulation and fabrication of data. The Commission will revert to this in more detail, however, including other players’ role in the continuation, see in particular Chapter 5.
Based on the account given above, the Commission finds that there is scientific dishonesty on the part of Jon Sudbø related to the PhD dissertation, New England Journal of Medicine 2001, Oncology 2001 and J Pathol 2001.