Proadess - Avaliação do Desempenho do Sistema de Saúde

Biblioteca (Todos)

Indicators of quality of care in general practices in England

AUTOR(ES): Jennifer Dixon, Emma Spencelayh, Anna Howells, Abraham Mandel, Felix Gille

ANO: 2015

RESUMO: In the summer of 2015 the Health Foundation was asked by the Secretary of State for Health to review indicators of the quality of care provided by general practices in England. We looked at how indicators could be developed to generate meaningful information that supports improvements to care and helps the public choose which practice might best meet their needs. Within the short timeframe available for the review (June to September 2015), the Foundation consulted with a wide range of organisations and individuals with an interest in how information about general practice care quality is collated and published. We assessed the literature, analysed indicators currently used as well as the websites on which they are published, and worked with insight agency BritainThinks to understand the public’s views. Our review focused on publication of indicators for the purposes of supporting local improvement of care, patient choice and voice, and the accountability and performance management of general practices. The review also strongly advises against making a composite score out of selected indicators to indicate the quality of care overall in general practice, or for particular population groups.

FONTE:

REFERENCIA: The Health Foundation, an independent health care charity, was asked by the Secretary of State for Health to review indicators of the quality of general practice in England. We looked at how they could be made better to support improvements to care, including how they are selected and presented. Within the short timeframe available for the review (June to September 2015), the Foundation consulted with a wide range of organisations and individuals with an interest in general practice and information. We assessed the available literature and analysed current indicators as well as the websites on which they are published. Our review focused on publication of indicators for the purposes of supporting local improvement of care, patient choice and voice, and the accountability and performance management of general practices. While our review was commissioned by the Department of Health, the Health Foundation did not receive any funding for completing the work. The Health Foundation retained full editorial control of the report’s content.

LINK

Random variation and rankability of hospitals using outcome indicators

AUTOR(ES): Van Dishoeck A, Lingsma HF, Mackenbach JP, et al

ANO: 2011

RESUMO: OBJECTIVE: There is a growing focus on quality and safety in healthcare. Outcome indicators are increasingly used to compare hospital performance and to rank hospitals, but the reliability of ranking (rankability) is under debate. This study aims to quantify the rankability of several outcome indicators of hospital performance currently used by the Dutch government. METHODS: From 52 indicators used by the Netherlands Inspectorate, the authors selected nine outcome indicators presenting a fraction and absolute numbers. Of these indicators, four were combined into two, resulting in seven indicators for analysis. The official data of 97 Dutch hospitals for the year 2007 were used. Uncertainty in the observed outcomes within the hospitals (within hospital variance, σ(2)) was estimated using fixed effect logistic regression models. Heterogeneity (between hospital variance, τ(2)) was measured with random effect logistic regression models. Subsequently, the rankability was calculated by relating heterogeneity to uncertainty within and between hospitals (τ(2)/(τ(2) +median σ(2))). RESULTS: Sample sizes varied but were typically around 200 per hospital (range of median 90-277) with a median of 2-21 cases, causing a substantial uncertainty in outcomes per hospital. Although fourfold to eightfold differences between hospitals were noted, the uncertainty within hospitals caused a poor (<50%) rankability in three indicators and moderate rankability (50-75%) in the other four indicators. CONCLUSION: The currently used Dutch outcome indicators are not suitable for ranking hospitals. When judging hospital quality the influence of random variation must be accounted for to avoid overinterpretation of the numbers in the quest for more transparency in healthcare. Adequate sample size is a prerequisite in attempting reliable ranking.

FONTE:

REFERENCIA: Van Dishoeck A, Lingsma HF, Mackenbach JP, et al. Random variation and rankability of hospitals using outcome indicators. BMJ Quality & Safety 2011;20:869-874.

LINK

Do all paths lead to Rome? Comparative analysis in the institutionalization of the evaluation

AUTOR(ES): Aquilino, N., Ballescá , M., Potenza, F., Rubio, J.

ANO: 2017

RESUMO: In recent years, evaluation has gained importance within public administrations of different countries. Even still, in many cases, it is implemented in a fragmented manner, in response to specific and isolated requests. On the other hand, to speak of a “system” with regards to monitoring and evaluation (M&E) implies that the practice derives from stable institutional arrangements that distribute the functions between those involved in the process, as well as other definitions with an orientation towards quality assessment information in a regular and sustained manner. This document analyzes different institutional arrangements of a set of countries with federal governments with respect to M&E. Those analyzed include Canada, Spain, Mexico, Brazil and Argentina. The case studies of the following sub-national governments are included as well: Catalonia, Jalisco, Pernambuco and the Autonomous City of Buenos Aires. Said studies demonstrate that if a single pattern of institutionalization of M&E functions does not exist, it is important to consider transversal aspects that affect its development. The formation of a system does not end with the creation of an agency or body responsible for M&E. For this organism to have the necessary capacities to carry out its functions, it requires an institutional insertion that guarantees a certain degree of political independence, while also enabling it to promote evaluations as a management tool. Likewise, it is necessary to establish rules defining responsibilities, rules that establish the obligation to evaluate and define quality standards in addition to defining a financing mechanism that guarantees the autonomy, perdurability of the system, and the commitments of evaluated programs or organisms. With respect to the evaluative practice, it is important to consider, not only the definition of an evaluative plan, but also quality control of information produced, and the promotion of evaluation, the use of which improves accountability, transparency of government actions, and decision making. This last facet requires the instauration of mechanisms promoting the use of information in management but also the generation of M&E information that could easily translate into decisions, improving programs and policies. The analyzed case studies demonstrate that M&E systems have started to expand to a level of sub-national states. These governments should act as an active partner in the design and implementation of the national M&E systems and promoters of their own systems. Because of this, the development of capacities and the institutionalization of areas in the government with these specific attributions from said matters are key.

FONTE:

REFERENCIA: Aquilino, N., Ballescá, M., Potenza, F., Rubio, J. (June de 2017). Do all paths lead to Rome? Comparative analysis in the institutionalization of the evaluation. Working Paper No. 159 Buenos Aires: CIPPEC.

LINK

Ranking hospitals: do we gain reliability by using composite rather than individual indicators?

AUTOR(ES): Hofstede SN, Ceyisakar IE, Lingsma HF, et al.

ANO: 2019

RESUMO: Background Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects ‘true’ hospital differences in outcomes versus random noise. Objective This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care. Methods Using the Dutch National Medical Registration (2007–2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (between-hospital variation/between-hospital+within hospital variation)×100% is classified as low (<50%), moderate (50%–75%) and high (>75%). Results Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%–96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%–95%). Conclusion Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators.

FONTE:

REFERENCIA: Hofstede SN, Ceyisakar IE, Lingsma HF, et al. Ranking hospitals: do we gain reliability by using composite rather than individual indicators? BMJ Quality & Safety 2019;28:94-102.

LINK

The multiple aims of pay-forperformance and the risk of unintended consequences

Trends in Health Equity in the United States by Race/Ethnicity, Sex, and Income, 1993-2017

AUTOR(ES): Zimmerman FJ, Anderson NW.

ANO: 2019

RESUMO: Importance Health equity is an often-cited goal of public health, included among the 4 overarching goals of the Department of Health and Human Services’ Healthy People 2020. Yet it is difficult to find summary assessments of national progress toward this goal. Objectives To identify variation in several measures of health equity from 1993 to 2017 in the United States and to test whether there are significant time trends. Design, Setting, and Participants Survey study using 25 years of data, from January 1, 1993, to December 31, 2017, from the Centers for Disease Control and Prevention’s Behavioral Risk Factor Surveillance System to assess trends in health equity and health justice by race/ethnicity, sex, and income in 3 categories by year. Main Outcomes and Measures Health equity was assessed separately for each of 2 health constructs: healthy days—the average of physical and mental healthy days over the previous 30 days—and general health in 5 categories, rescaled to approximate a continuous variable. For each health construct, average health was calculated along with 4 measures of health equity: disparities across 3 income groups; black-white disparities; health justice—a measure of the correlation of health outcomes with income, race/ethnicity and sex; and a summary health equity metric. Results Among the 5 456 006 respondents, the mean (SD) age was 44.5 (12.7) years; 3 178 688 (58.3%) were female; 4 163 945 (76.3%) were non-Latinx white; 474 855 (8.7%) were non-Latinx black; 419 542 (7.7%) were Latinx; and 397 664 (7.3%) were of other race/ethnicity. The final sample included 5 456 006 respondents for self-reported health and 5 349 527 respondents for healthy days. During the 25-year period, the black-white gap showed significant improvement (year coefficient: healthy days, 0.021; 97.5% CI, 0.012 to 0.029; P < .001; self-reported health, 0.030; 97.5% CI, 0.025 to 0.035; P < .001). The health equity metric for self-reported health showed no significant trend. For healthy days, the Health Equity Metric declined over time (year coefficient: healthy days, −0.025; 97.5% CI, −0.033 to −0.017; P < .001). Health justice declined over time (year coefficient: healthy days, −0.045; 97.5% CI, −0.053 to −0.038; P < .001; self-reported health, −0.035; 97.5% CI, −0.046 to−0.023; P < .001), and income disparities worsened (year coefficient: healthy days, −0.060; 97.5% CI, −0.076 to −0.044; P < .001; self-reported health, −0.029; 97.5% CI, −0.046 to −0.012; P < .001). Conclusions and Relevance Results of this analysis suggest that there has been a clear lack of progress on health equity during the past 25 years in the United States. Achieving widely shared goals of improving health equity will require greater effort from public health policy makers, along with their partners in medicine and the sectors that contribute to the social determinants of health.

FONTE:

REFERENCIA: Zimmerman FJ, Anderson NW. Trends in Health Equity in the United States by Race/Ethnicity, Sex, and Income, 1993-2017. JAMA Netw Open. 2019;2(6):e196386. doi:10.1001/jamanetworkopen.2019.6386

LINK

Health at a Glance 2019

Performance of UK National Health Service compared with other high income countries: observational study

AUTOR(ES): Irene Papanicolas, Elias Mossialos, Anders Gundersen, Liana Woskie, Ashish K Jha

ANO: 2019

RESUMO: Objective - To determine how the UK National Health Service (NHS) is performing relative to health systems of other high income countries, given that it is facing sustained financial pressure, increasing levels of demand, and cuts to social care. Design - Observational study using secondary data from key international organisations such as Eurostat and the Organization for Economic Cooperation and Development. Setting Healthcare systems of the UK and nine high income comparator countries: Australia, Canada, Denmark, France, Germany, the Netherlands, Sweden, Switzerland, and the US. Main outcome measures 79 indicators across seven domains: population and healthcare coverage, healthcare and social spending, structural capacity, utilisation, access to care, quality of care, and population health. Results - The UK spent the least per capita on healthcare in 2017 compared with all other countries studied (UK $3825 (£2972; €3392); mean $5700), and spending was growing at slightly lower levels (0.02% of gross domestic product in the previous four years, compared with a mean of 0.07%). The UK had the lowest rates of unmet need and among the lowest numbers of doctors and nurses per capita, despite having average levels of utilisation (number of hospital admissions). The UK had slightly below average life expectancy (81.3 years compared with a mean of 81.7) and cancer survival, including breast, cervical, colon, and rectal cancer. Although several health service outcomes were poor, such as postoperative sepsis after abdominal surgery (UK 2454 per 100 000 discharges; mean 2058 per 100 000 discharges), 30 day mortality for acute myocardial infarction (UK 7.1%; mean 5.5%), and ischaemic stroke (UK 9.6%; mean 6.6%), the UK achieved lower than average rates of postoperative deep venous thrombosis after joint surgery and fewer healthcare associated infections. Conclusions - The NHS showed pockets of good performance, including in health service outcomes, but spending, patient safety, and population health were all below average to average at best. Taken together, these results suggest that if the NHS wants to achieve comparable health outcomes at a time of growing demographic pressure, it may need to spend more to increase the supply of labour and long term care and reduce the declining trend in social spending to match levels of comparator countries.

FONTE:

REFERENCIA: Papanicolas Irene, Mossialos Elias, Gundersen Anders, Woskie Liana, Jha Ashish K. Performance of UK National Health Service compared with other high income countries: observational study BMJ 2019; 367 :l6326

LINK

Heterogeneity in enterotoxigenic Escherichia coli and shigella infections in children under 5 years of age from 11 African countries: a subnational approach quantifying risk, mortality, morbidity, and stunting

AUTOR(ES): Karoun H Bagamian, John D Anderson, Farzana Muhib, Oliver Cumming, Lindsey A Laytner, Thomas F Wierzba, Richard Rheingans

ANO: 2020

RESUMO: Background - Diarrhoea, a global cause of child mortality and morbidity, is linked to adverse consequences including childhood stunting and death from other diseases. Few studies explore how diarrhoeal mortality varies subnationally, especially by cause, which is important for targeting investments. Even fewer examine indirect effects of diarrhoeal morbidity on child mortality. We estimated the subnational distribution of mortality, morbidity, and childhood stunting attributable to enterotoxigenic Escherichia coli (ETEC) and shigella infection in children younger than 5 years from 11 eastern and central African countries. These pathogens are leading causes of diarrhoea in young children and have been linked to increased childhood stunting. Methods - We combined proxy indicators of morbidity and mortality risk from the most recent Demographic and Health Surveys with published relative risks to estimate the potential distribution of diarrhoeal disease risk. To estimate subnational burden, we used country-specific or WHO region-specific morbidity and mortality estimates and distributed them subnationally by three indices that integrate relevant individual characteristics (ie, underweight, probability of receiving oral rehydration treatment of diarrhoea, and receiving vitamin A supplementation) and household characteristics (ie, type of drinking water and sanitation facilities). Findings - Characterising ETEC and shigella subnational estimates of indirect morbidity (infection-attributable stunting) and indirect mortality (stunting-related deaths from other infectious diseases) identified high-risk areas that could be missed by traditional metrics. Burundi and Democratic Republic of the Congo had the highest ETEC-associated and shigella-associated mortality and stunting rates. Mozambique, Democratic Republic of the Congo, and Zimbabwe had the greatest subnational heterogeneity in most ETEC and shigella mortality measures. Inclusion of indirect ETEC and shigella mortality in burden estimates resulted in a 20–30% increase in total ETEC and shigella mortality rates in some subnational areas. Interpretation - Understanding the indirect mortality and morbidity of diarrhoeal pathogens on a subnational level will strengthen disease control strategies and could have important implications for the relative impact and cost-effectiveness of new enteric vaccines. Because our methods rely on publicly available data, they could be employed for national planning.

FONTE:

REFERENCIA: Karoun H Bagamian, John D Anderson, Farzana Muhib, Oliver Cumming, Lindsey A Laytner, Thomas F Wierzba, Richard Rheingans, Heterogeneity in enterotoxigenic Escherichia coli and shigella infections in children under 5 years of age from 11 African countries: a subnational approach quantifying risk, mortality, morbidity, and stunting, The Lancet Global Health, Volume 8, Issue 1, 2020, Pages e101-e112, ISSN 2214-109X, https://doi.org/10.1016/S2214-109X(19)30456-5. (http://www.sciencedirect.com/science/article/pii/S2214109X19304565)

LINK

Desigualdades regionais e sociais em saúde segundo inquéritos domiciliares (Brasil, 1998-2013)

AUTOR(ES): Viacava Francisco, Porto Silvia Marta, Carvalho Carolina de Campos, Bellido Jaime Gregó rio.

ANO: 2019

RESUMO: Este artigo busca discutir a evolução das desigualdades em saúde e no acesso aos serviços de saúde nas grandes regiões a partir de inquéritos domiciliares realizados de 1998 a 2013. As desigualdades sociais foram analisadas pelas razões de extremos de anos de escolaridade, considerando duas faixas etárias (18 a 59 anos e 60 anos ou mais). Nas condições de saúde, observa-se, nos dois grupos etários, uma pior avaliação do estado de saúde e um aumento da prevalência de diabetes e hipertensão, o que pode estar relacionado à expansão da atenção básica. Quanto à realização de consultas médicas no último ano, encontra-se, no geral, maior acesso, com manutenção de pequenas desigualdades. A maior utilização de consulta odontológica entre os de menor escolaridade provoca uma redução nas desigualdades, que ainda são significativas. As internações hospitalares, ao longo da série, são maiores entre os menos escolarizados, e há uma redução nas taxas nos dois grupos etários, em quase todas as regiões. Percebe-se um aumento na realização de mamografia por mulheres menos escolarizadas, com diminuição da desigualdade. Os resultados corroboram a necessidade da continuidade dos inquéritos domiciliares para o monitoramento das desigualdades regionais e sociais no acesso ao sistema de saúde.

FONTE:

REFERENCIA: VIACAVA, Francisco et al . Desigualdades regionais e sociais em saúde segundo inquéritos domiciliares (Brasil, 1998-2013). Ciênc. saúde coletiva, Rio de Janeiro , v. 24, n. 7, p. 2745-2760, July 2019 . Available from <http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232019000702745&lng=en&nrm=iso>. access on 07 Feb. 2020. Epub July 22, 2019. http://dx.doi.org/10.1590/1413-81232018247.15812017.

LINK

Indicadores hospitalares de acesso e efetividade e crise econômica: análise baseada nos dados do Sistema Único de Saúde, Brasil e estados da região Sudeste, 2009-2018

AUTOR(ES): Martins Mô nica, Lima Sheyla Maria Lemos, Andrade Carla Lourenç o Tavares de, Portela Margareth Crisó stomo.

ANO: 2019

RESUMO: No contexto de crise e restrições de recursos é razoável supor o agravamento de fragilidades do Sistema Único de Saúde (SUS), como desigualdades regionais, subfinanciamento e problemas na qualidade do cuidado. Este estudo explorou a aplicação de indicadores de acesso e efetividade, facilmente compreensíveis e calculados, passíveis de refletir a crise na rede hospitalar. Cinco indicadores extraídos do Sistema de Informações Hospitalares, relativos ao Brasil e a estados da Região Sudeste, foram analisados no período de 2009-2018: internações resultantes em morte; internações cirúrgicas resultantes em morte; cirurgias eletivas no total das internações cirúrgicas; próteses de quadril na população de idosos; e angioplastias na população de 20 anos ou mais. Utilizaram-se gráficos de controle estatístico para a comparação dos indicadores entre estados, antes e a partir de 2014. No Brasil, as mortes hospitalares tiveram um leve crescimento enquanto que as mortes cirúrgicas uma queda; as cirurgias eletivas e próteses de quadril também diminuíram. No Sudeste, o Rio de Janeiro apresentou os piores resultados, em especial a queda de cirurgias eletivas. Os resultados ilustram o potencial dos indicadores para monitorar efeitos da crise sobre o cuidado hospitalar.

FONTE:

REFERENCIA: MARTINS, Mônica et al . Indicadores hospitalares de acesso e efetividade e crise econômica: análise baseada nos dados do Sistema Único de Saúde, Brasil e estados da região Sudeste, 2009-2018. Ciênc. saúde coletiva, Rio de Janeiro , v. 24, n. 12, p. 4541-4554, Dec. 2019 . Available from <http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232019001204541&lng=en&nrm=iso>. access on 07 Feb. 2020. Epub Nov 25, 2019. http://dx.doi.org/10.1590/1413-812320182412.25262019.

LINK

Prevalência de utilização de serviços de saúde no Brasil: revisão sistemática e metanálise

Composite measures of healthcare quality: sensible in theory, problematic in practice

Bases para um debate sobre a reforma hospitalar do SUS: as necessidades sociais e o dimensionamento e tipologia de leitos hospitalares em um contexto de crise de acesso e qualidade

AUTOR(ES): Armando Antonio De Negri Filho

ANO: 2016

RESUMO: INTRODUÇÃO: Esta tese trata da discussão em torno às perguntas necessárias para compor uma agenda política estratégica da reforma hospitalar brasileira. Buscou-se construir argumentos sobre a relevância de abordar nessas perguntas a grande insuficiência da oferta de leitos hospitalares, como indicador da falta de resposta oportuna em volume e qualidade para materializar a resposta às necessidades da população e garantir os direitos humanos e sociais à saúde, conforme os preceitos constitucionais de 1988. MÉTODOS: O estudo adotou um enfoque desde a perspectiva de política de saúde, o que nos remete ao debate sobre processo e poder, destacando os ciclos de formulação e implementação de políticas, a hierarquização desses debates como de política estratégica (High Politics) ou setorial (Low Politics) e as condições requeridas em termos de atores e espaços de decisão a considerar. Realizou-se a revisão da experiência internacional para identificar as referências do número e perfil de leitos requeridos para atender populações nacionais em seus territórios e também foram resgatados os parâmetros adotados no Brasil desde o Estado Novo. Para a análise da oferta hospitalar entre 2005 e 2014, foram utilizados os dados secundários extraídos do Cadastro Nacional de Estabelecimentos em Saúde - CNES, do Sistema de Informações Ambulatoriais e Hospitalares - SIA / SIH, da Agencia Nacional de Saúde Suplementar - ANS e do Instituto Brasileiro de Geografia e Estatística - IBGE. RESULTADOS: 1. O número e tipo de leitos necessários e o número, porte, tipo e características sistêmicas dos serviços hospitalares que os abrigariam, são temas que se consideraram relevantes para compor as perguntas para a agenda estratégica em debate. 2. A análise da política do período 2004 a 2014 permite identificar a fragmentação e descontinuidade das ações mesmo obtendo resultados pontuais de melhora, mas afastando-se da promessa estratégica de algumas agendas ensaiadas no período. 3. Ganhou destaque a diminuição continuada do estoque geral de leitos e de leitos para o SUS, alcançando níveis muito inferiores a qualquer parâmetro comparativo de outros sistemas universais de saúde, particularmente quando adotou-se o critério de leitos efetivos para examinar a oferta de leitos e seu desempenho. Na Saúde Suplementar houve aumento de leitos, porém queda no número por mil assegurados dado o crescimento das coberturas hospitalares no período. CONCLUSÕES: A crise persistente de acesso hospitalar constitui razão para construir uma agenda estratégica para a reforma hospitalar brasileira. Ao buscar o caminho para elevar o tema da crise hospitalar a um tratamento de alta política, foram mapeados: o conteúdo de política hospitalar dos planos estaduais de saúde, documentos técnicos estaduais e organogramas de coordenação da atenção hospitalar estadual, examinando-se a possibilidade dos governos estaduais liderarem no processo de regionalização a construção de uma agenda política estratégica estadual e nacional, fortalecendo o diálogo entre os entes federados, incluindo seus próprios prestadores e os prestadores não estatais particularmente os filantrópicos, além de estender o diálogo com o corpo médico e os usuários entendidos como cidadãos sujeitos de direitos. Para alicerçar este caminho possível se apresenta o processo de elaboração participativa de planos diretores de redes e hospitais por estados e suas regiões, como forma de construção de comunidades epistêmicas e suas projeções para a sustentação de uma agenda para a alta política, orientada a alcançar 4 leitos por 1000 habitantes em 20 anos de esforço sustentado

FONTE:

REFERENCIA: De Negri Filho AA. Bases para um debate sobre a reforma hospitalar do SUS: as necessidades sociais e o dimensionamento e tipologia de leitos hospitalares em um contexto de crise de acesso e qualidade [tese]. São Paulo: Universidade de São Paulo; 2016.

LINK

Health at a Glance 2017: OECD Indicators

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16