first_imgAn Indonesian scientist at work on Borneo. The country’s new index seeks to capture academics’ performance. Surging ahead New incentives have driven a sharp rise in Scopus-indexed papers from Indonesia. National Geographic Image Collection/Alamy Stock Photo By Dyna Rochmyaningsih Jan. 8, 2019 , 5:15 PM Click to view the privacy policy. Required fields are indicated by an asterisk (*) JAKARTA—Last July, when Indonesia’s Ministry of Research, Technology and Higher Education (RISTEK) here honored eight researchers, along with institutions and journals, for their exceptional contributions to science, observers noticed something odd. Many of the laureates were relatively unknown academics from second-tier universities; underdogs had apparently become leaders.It didn’t take curious scientists long to figure out why. The honors went to top scorers in Indonesia’s Science and Technology Index (SINTA), a system introduced in early 2017 to measure research performance. Critics showed that several winners had inflated their SINTA score by publishing large numbers of papers in low-quality journals, citing their own work excessively, or forming networks of scientists who cited each other.It’s unclear whether formal rules were broken, but SINTA’s architects concede they were outwitted. And the revelations have led to a fierce discussion about SINTA, a unique nationwide attempt to capture the output of every academic in a single formula. Some say it should not be used to produce rankings, or should even be abandoned. But the government is undeterred: After a meeting on 3–4 January, it announced the rollout later this year of an improved version. SINTA “gives recognition to Indonesian scientists, triggers competition among them, and motivates them to be better,” says Sadjuga, RISTEK’s director of intellectual property management. (Like many Indonesians, he goes by only one name.) Country * Afghanistan Aland Islands Albania Algeria Andorra Angola Anguilla Antarctica Antigua and Barbuda Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia, Plurinational State of Bonaire, Sint Eustatius and Saba Bosnia and Herzegovina Botswana Bouvet Island Brazil British Indian Ocean Territory Brunei Darussalam Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos (Keeling) Islands Colombia Comoros Congo Congo, the Democratic Republic of the Cook Islands Costa Rica Cote d’Ivoire Croatia Cuba Curaçao Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Falkland Islands (Malvinas) Faroe Islands Fiji Finland France French Guiana French Polynesia French Southern Territories Gabon Gambia Georgia Germany Ghana Gibraltar Greece Greenland Grenada Guadeloupe Guatemala Guernsey Guinea Guinea-Bissau Guyana Haiti Heard Island and McDonald Islands Holy See (Vatican City State) Honduras Hungary Iceland India Indonesia Iran, Islamic Republic of Iraq Ireland Isle of Man Israel Italy Jamaica Japan Jersey Jordan Kazakhstan Kenya Kiribati Korea, Democratic People’s Republic of Korea, Republic of Kuwait Kyrgyzstan Lao People’s Democratic Republic Latvia Lebanon Lesotho Liberia Libyan Arab Jamahiriya Liechtenstein Lithuania Luxembourg Macao Macedonia, the former Yugoslav Republic of Madagascar Malawi Malaysia Maldives Mali Malta Martinique Mauritania Mauritius Mayotte Mexico Moldova, Republic of Monaco Mongolia Montenegro Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands New Caledonia New Zealand Nicaragua Niger Nigeria Niue Norfolk Island Norway Oman Pakistan Palestine Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Qatar Reunion Romania Russian Federation Rwanda Saint Barthélemy Saint Helena, Ascension and Tristan da Cunha Saint Kitts and Nevis Saint Lucia Saint Martin (French part) Saint Pierre and Miquelon Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Sint Maarten (Dutch part) Slovakia Slovenia Solomon Islands Somalia South Africa South Georgia and the South Sandwich Islands South Sudan Spain Sri Lanka Sudan Suriname Svalbard and Jan Mayen Swaziland Sweden Switzerland Syrian Arab Republic Taiwan Tajikistan Tanzania, United Republic of Thailand Timor-Leste Togo Tokelau Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Turks and Caicos Islands Tuvalu Uganda Ukraine United Arab Emirates United Kingdom United States Uruguay Uzbekistan Vanuatu Venezuela, Bolivarian Republic of Vietnam Virgin Islands, British Wallis and Futuna Western Sahara Yemen Zambia Zimbabwe Sign up for our daily newsletter Get more great content like this delivered right to you! Country (GRAPHIC) M. ENSERINK/SCIENCE; (DATA) RISTEK Indonesia has introduced several other policies in the past 6 years to boost research output from its more than 250,000 academics, who work at more than 4000 universities. University professors may lose almost half of their salary if they don’t publish in international journals, for instance. As a result, the number of papers published by authors in Indonesia has soared from just under 7000 in 2014 tomore than 28,000 last year, according to Scopus, a database operated by Dutch publisher Elsevier. Indonesia seems set to overtake Malaysia as the region’s biggest research producer by 2020.SINTA—also the name of a Sanskrit goddess—turned the pressure up a notch. It combines data from Scopus and Google Scholar with information submitted by Indonesian academics to track published papers, citations, and researchers’ h-index, a controversial metric reflecting both output quantity and citations. These numbers are used to calculate a personal score that is taken into account when academics apply for research grants; a high score may also help with promotions and salary negotiations.Many other countries use publication and citation data to evaluate research; some pay hefty cash bonuses for papers in top-tier journals. But, “There is nothing like [SINTA] that I know of,” says Diana Hicks, a research metrics expert at the Georgia Institute of Technology in Atlanta. The extra push was welcome, says Danang Birowosuto, an Indonesian physicist at CINTRA, an international research group in Singapore: “Our international competence in science is still very low.” But many Indonesian academics worried that SINTA might harm their reputations. Thousands joined groups on social media to help each other navigate the new numbers-driven landscape. “Although the original aim was sincere,” discussions soon turned to gaming the system, says plant biologist Andik Wijayanto of the State University of Malang. Email How to shine in Indonesian science? Game the system In October 2018, Anis Fuad, a health informatician at Gadjah Mada University in Yogyakarta, presented RISTEK with a detailed analysis of the problems. Indonesia’s most-cited 2018 paper so far wasn’t a major breakthrough, Fuad noted, but a study titled “Analysis of Student Satisfaction Toward Quality of Service Facility,” presented at a workshop co-organized by the Indonesian Publications Collaboration Community (KO2PI) and published in conference proceedings, a type of publication that gets minimal peer review. The study had been cited 42 times, often in papers on unrelated topics—including mosque architecture and cold storage of fish—that were also published in conference series or in low-quality open-access journals no longer indexed in Scopus.One of the paper’s 10 authors was statistician Ansari Saleh Ahmar of the State University of Makassar, who won SINTA awards in two categories last July; he co-authored more than 100 papers in 2017 and 2018 and has been cited almost 600 times. Ahmar is also president of KO2PI, which has run workshops in an extraordinary range of scientific fields. On a poster produced in early 2017, KO2PI promised participants a paper in a Scopus-indexed proceeding in return for a 1.5 million rupiah ($106) fee. Ahmar says he was “surprised” by his own citation rate, but says statistical papers are often cited in seemingly unrelated fields. He says he is no longer active in KO2PI and, given the controversy, would now like to return his award.After asking Ahmar and other academics suspected of gaming the system for an explanation, RISTEK has deleted their SINTA accounts, Sadjuga says, but it has not withdrawn the awards because “the public shaming is punishment enough.” Sadjuga says problematic data in Scopus and scientists’ unethical behavior contributed to the problem but does not blame SINTA itself. (An Elsevier spokesperson says Scopus has stopped indexing three journals that many Indonesian scientists have published in and is investigating “concerns” about the conference series used by KO2PI, which is published by the U.K. Institute of Physics.)Gaming aside, Indonesia’s research evaluation should not rely on a commercial database, says Dasapta Erwin Irawan, a hydrogeologist at Bandung Institute of Technology. He also says the system’s preference for Scopus-indexed international journals is misguided, because research in Indonesian journals may be just as good and sometimes more relevant. RISTEK doesn’t entirely ignore local journals: It has created an online portal, named Garuda, to more than 7000 journals in the Indonesian language, as well as a journal accreditation system. But researchers win far fewer SINTA points when papers in local journals are cited and none at all for publishing in them.That lack of appreciation for locally relevant research violates the “Leiden Manifesto for research Metrics,” an influential paper Hicks and three co-authors published in 2015. Hicks says SINTA falls short on several other principles in the manifesto, which stipulates that metrics should “support a qualitative, expert assessment” and “account for variation by field in publication and citation practices.” SINTA currently does neither.A new version of SINTA, set to be launched this year, will integrate data from several additional sources, including the Web of Science and the Indonesian National Library. It will also give researchers credit for other types of output, such as books, artwork, and patents. A new tool will flag self-citation and the ministry will disseminate scientific integrity guidelines to Indonesian universities.But Mikrajuddin Abdullah, a physicist at Bandung Institute of Technology, says RISTEK should still review last year’s awards and retract them if they were based on misconduct: “It will teach us that scientific achievement does not come suddenly, but is the result of a long period of perseverance.”last_img read more