SciELO - Scientific Electronic Library Online

 
vol.60 número1The tangible consequences and intangible implications of laparoscopic cholecystectomy-associated bile duct injuries índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Em processo de indexaçãoSimilares em Google

Compartilhar


South African Journal of Surgery

versão On-line ISSN 2078-5151
versão impressa ISSN 0038-2361

S. Afr. j. surg. vol.60 no.1 Cape Town Mar. 2022

http://dx.doi.org/10.17159/2078-5151/2022/v60n1a3553 

GENERAL SURGERY

 

The negative appendicectomy rate at Charlotte Maxeke Johannesburg Academic Hospital - a 10-year review

 

 

SJ GrossbergI; JD BrenerI; EL BizosI; GP CandyII; DB BizosI, II

IDepartment of Surgery, Charlotte Maxeke Johannesburg Academic Hospital, South Africa
IIDepartment of Surgery, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, South Africa

Correspondence

 

 


ABSTRACT

BACKGROUND: The negative appendicectomy rate (NAR) is defined as the proportion of surgically removed appendices that are pathologically normal. The acceptable NAR has been a debated issue. Previously, a higher rate was accepted, whilst newer thinking favours a lower rate. Diagnosing appendicitis is often a clinical challenge and may require cross-sectional imaging to assist in the diagnosis
METHODS: A retrospective review was conducted at the Charlotte Maxeke Johannesburg Academic Hospital. Appendix histopathological reports were retrieved for patients older than 18 years over a 10-year period. Reports of ultrasound (US) and/or computed tomography (CT) scans were analysed in the last 18 months
RESULTS: One thousand two hundred and seventeen appendicectomy specimens were included. The overall NAR was 19%. This demonstrated a significant downward trend over the period (p < 0.003). Per gender, the female NAR showed a significant decline (p = 0.002) while the male decline was not significant (p = 0.517). Reproductive-age females were found to have significantly higher NAR as compared to other age groups. The overall perforation rate was 17% which demonstrated a significant increase over the study period (p = 0.012). In the last 18 months, 240 appendicectomies were performed. One hundred and eleven patients underwent imaging (46%), of which 78 underwent ultrasound (70%), 14 CT (13%) and 19 US and CT (17%
CONCLUSION: The overall NAR declined significantly over the period. Females under the age of 45 were found to have significantly higher NARs. Further prospective studies are needed to determine the benefit and feasibility of preoperative CT in resource-limited settings, particularly in reproductive-age females to reduce the NAR

Keywords: appendicitis, negative appendicectomy


 

 

Introduction

Acute appendicitis is one of the most common causes of an acute surgical abdomen.1 The diagnosis may be difficult as the presentation can be atypical and the differential diagnosis for an acute abdomen encompasses a variety of abdominal and pelvic organ pathologies.2 A good history, clinical examination with appropriate biochemical and radiological investigations are needed to make the diagnosis.1 The gold standard in confirming the diagnosis of appendicitis is the histopathological examination of the appendix specimen.3 Early diagnosis is crucial for timeous treatment and to prevent complications.4 There is a balance between early surgical intervention to prevent complications and having a more conservative approach to reduce unnecessary surgical procedures and the complications thereof. 5

The negative appendicectomy rate (NAR) or 'lily-white' appendix rate can be defined as 'the proportion of pathologically normal appendices removed surgically in patients suspected of having acute appendicitis'.6 A wide range of NAR has been reported in the literature.7 The acceptable NAR is debated. Previously a higher NAR was viewed as acceptable as it was assumed to reduce the number of perforations and the associated complications.8 This was balanced against the complications of an appendicectomy for a normal appendix. Flum and Koepsell challenged this idea, demonstrating that a negative appendicectomy was not as innocuous as was previously thought. They demonstrated that a negative appendicectomy was associated with a longer length of stay and higher cost of admission, case fatality rate, and infection rate.9

Imaging plays a fundamental role in diagnosing acute appendicitis. Abdominal and pelvic ultrasound and computerised tomography (CT) scanning are most commonly used. Ultrasound is associated with no radiation but is operator dependent. A 2016 meta-analysis by Giljaca et al. demonstrated that ultrasound has a sensitivity of 69% (95% confidence interval [CI] 59-78%) and a specificity of 81% (95% CI 73-88%). The post-test probability of a positive test was 92% (95% CI 88-95%) and a negative test was 55% (95% CI 46-63%).10

CT is more accurate than ultrasound. A 2019 Cochrane review demonstrated a sensitivity of 0.95 (95% CI 0.930.96) and a specificity of 0.94 (95% CI 0.92-0.95). The probability of having appendicitis following a positive CT result was 0.92 (95% CI 0.90-0.94), while the probability of having appendicitis following a negative CT result was 0.04 (95% CI 0.03-0.05).11 There is conflicting evidence on whether preoperative CT scan use significantly affects the NAR.12

Charlotte Maxeke Johannesburg Academic Hospital (CMJAH) is a tertiary referral hospital with direct emergency admissions and referrals from cluster hospitals. The policy of the surgical unit is that unless there is diffuse peritonitis, US is recommended for female patients with suspected acute appendicitis. Male patients with typical signs and symptoms are taken at the surgeon's discretion directly to theatre. Imaging is reserved for cases with diagnostic uncertainty. CT is performed in both male and female patients for complicated cases, diagnostic uncertainty or if a mass is palpable. Diagnostic laparoscopy is used when imaging fails to confirm a diagnosis.

The appendix is always removed if a right iliac fossa incision is made. If approached laparoscopically, it is at the surgeon's discretion whether to remove a normal looking appendix.

The primary objective of this study was to determine the NAR in adults at the CMJAH and to assess trends over ten years. Secondary objectives were to analyse the value of imaging modalities performed in the last 18 months of the study period.

 

Methods

A retrospective review was conducted in the Department of Surgery at the CMJAH. Appendix histopathological specimen reports over 10 years (1 June 2008-31 May 2018) were retrieved from the National Health Laboratory Service. Inclusion criteria: patients > 18 years of age who underwent an appendicectomy for acute appendicitis. Exclusion criteria: patients < 18 years of age, incidental appendicectomy, interval appendicectomy and pregnant patients. Clinical information was obtained, where available, from the clinical history on the pathology reports.

The diagnosis of acute appendicitis was defined histologically as per the pathology report. Incidental appendicectomy was defined as the removal of the appendix that occurred during non-appendiceal surgery. Interval appendicectomy was defined as the removal of the appendix after conservative management had been instituted. Reproductive-age females were taken to be those less than 45 years of age.

The following variables were collected: age, gender, date of procedure, the presence/absence of acute appendicitis and whether perforation was present.

Radiological records were retrieved from the picture archiving and communication system (PACS) in the last 18 months (the system was introduced at CMJAH in mid-2016). Information on whether the patients underwent an ultrasound and/or CT scan was extracted, the imaging reports where available were analysed.

 

Statistics

Data were recorded in Microsoft Excel and data were presented as n (frequency) or percentage and as mean ± standard deviation (SD) as applicable, in graphs. Statistical comparisons between groups were determined using Statistica V13.5. A chi-squared test or a Fisher's exact test were used to compare frequencies and trends over time were determined from the correlation coefficient. A calculated p-value of < 0 .05 between groups was regarded as being of statistical significance.

 

Results

One thousand two hundred and seventeen appendicectomy specimens were included in the study. There were 788 male patients and 428 females (1 missing gender). The overall NAR over the 10 years was 19%. The NAR showed a significant downward trend over the study period (p < 0.003) (Figure 1). On analysis of the trends per gender, the female NAR showed a significant decline over the study period (p = 0.002), while the male decline was not significant (p = 0.517) (Figures 2 and 3). Reproductive-age females were found to have a significantly higher NAR as compared to other age groups.

 

 

 

 

 

 

The overall perforation rate over the study period was 17%. This rate showed a significant increase over the study period (p = 0.012) (Figure 4). When analysing the under 45 age group, a significant increase in the perforation rate was noted in male patients (p = 0.008), while females demonstrated a downward trend which was not significant (p = 0.34).

 

 

In the last 18 months of the study period, 240 appen-dicectomies were performed. One hundred and eleven patients underwent imaging (46%), of which 78 underwent ultrasound (70%), 14 CT (13%) and 19 US and CT (17%).

All 33 CT scans had reports with 25/97 (26%) ultrasound reports missing. Fifty-five out of 72 (76%) reported ultrasounds and 27/33 (82%) CTs were positive. Only 33/55 (60%) females with non-perforated appendicitis had an US.

 

Discussion

To our knowledge, this is the largest single centre review conducted in South Africa looking at the NAR. Our overall NAR over the ten years was 19% (ranging from 14-29.2%). This was comparable to other South African centres (Table I).13-26 A South African systematic review demonstrated an overall NAR of 17%.27 As compared to international centres, a multicentre prospective study in Australia demonstrated a NAR of 19%.28 A nationwide audit in the Netherlands revealed a rate of 2.2%.29 In the United Kingdom, a NAR of 20.6% was reported.30 In the United States of America, data from the American College of Surgeons National Surgical Quality Improvement Project (ACS-NSQIP) database revealed a NAR of 4.5%, while single centre data have reported rates as low as 1.7%.6,7

Our study demonstrated a significant decline in the NAR over the study period. Both the male and female rates declined over the period, although only the female decline was significant. Possible explanations for the reduced overall rate include improved clinical acumen of the clinician and delayed presentation. The latter is possible in our study as we found a significant increase in the perforation rate over the study period. The delayed presentation may lead to perforation and consequently a more severe clinical presentation which may assist with diagnosis. As our institution is a tertiary centre, the referral process may result in a delay in the diagnosis and treatment of the patient. Other possible explanations for the reduced rate include possible increased use of laparoscopic surgery (normal-appearing appendices may not be removed) and increased use of imaging. At this time, using our current variables we are unable to definitively explain what contributed to the decline in the NAR over the study period and why there was a difference between male and female patients.

The rising perforation rate in the study is concerning. From our current data, we are unable to explain this. Possible reasons are delayed presentation to referral hospital, delayed presentation to our institution, late referrals, delay in transfer, delay in investigations and delay to surgery.

Cross-sectional imaging, especially CT scans, is viewed as a useful modality to assist in diagnosing acute appendicitis. CT is associated with high sensitivity and specificity for diagnosing acute appendicitis.11 Despite this, there is conflicting evidence in the literature as to whether preoperative CT scanning is associated with a reduction in the NAR.12 Examples include Cuschieri et al., who in their review of 3 540 cases demonstrated a significant decrease in the NAR over time and a significantly increased use of preoperative imaging.31 Raja et al. in their single centre review also demonstrated a significant reduction in the NAR to 1.7%, which correlated with a significant increase in preoperative CT use.6 Tseng et al. in their review of 11 841 patients in the ACS-NSQIP database revealed that preoperative CT imaging demonstrated the lowest odds of a negative appendix.7 Drake et al. analysed Surgical Care and Outcomes Assessment Programme data from 19 327 patients and demonstrated a significantly lower NAR in those who underwent preoperative imaging.32

Conversely, Musunuru et al. in their review of 411 patients found no difference in the NAR between those who underwent preoperative CT imaging and those who did not.33 Perez et al. in their single centre review showed that CT did not affect the NAR.34 Frei et al. in a multicentre review, demonstrated an increasing use of CT but no significant decline in the NAR.35 Vadeboncoeur et al. demonstrated no decrease in the NAR with preoperative CT imaging.36 Flum et al. analysing data from the Group Health Cooperative demonstrated that the NAR did not change over time despite the introduction of US/CT, although they attributed this to the low sensitivity of CT in their study.37

Females have been shown to benefit from preoperative CT scanning.3839 Morse et al. found that reproductive-age females demonstrated a significant decline in the NAR with preoperative CT imaging.40 Coursey et al. demonstrated that the increased use of preoperative CT imaging in females under 45 years of age coincided with a lower NAR.12

Our finding that reproductive-age females had a significantly higher NAR when compared to other age groups can possibly be attributed to multiple gynaecological pathologies which can confound the diagnosis in this patient population.12 This age group has been associated with a higher NAR.3

A prospective study is needed to comprehensively assess factors contributing to the NAR in our centre in order to be able to implement effective strategies to reduce the NAR. Liberal preoperative CT scanning needs to be evaluated in terms of benefit, feasibility and cost-effectiveness in a low resource environment. Due to the nature of our setting, it may not be feasible to perform a CT scan for all patients, and therefore practical guidelines would need to be extrapolated as to which group of patients would benefit most from preoperative cross-sectional imaging. From our study and the literature, reproductive-age females appear to be a high-risk group and thus a possible target for initial intervention. As this group has shown to benefit from preoperative CT scanning in the literature, it may be advisable to recommend that all reproductive-age females who present with suspected appendicitis at our centre undergo CT scanning. This will aid in the diagnostic process and may assist in potentially reducing the NAR. Other options to be considered include point of care ultrasound and diagnostic laparoscopy.

Our study was limited due to its retrospective nature. Clinical notes were not retrieved, and pathology slides were not reviewed, and therefore findings were reliant on the pathology and radiology reports. There was also no standardisation in both the pathology and radiology reports. The clinical history was often poor on the pathology reports. There were also limited variables examined, some of which may have been able to account for our results. Due to the limited imaging numbers, we are reluctant to draw conclusions.

 

Conclusion

Our study demonstrated a significant decrease in the NAR over the study period. Our rate, although higher than selected international centres, is comparable to other South African and international centres. The reason for the decline is not evident from our current variables but may be due to later presentation and an increased perforation rate. Reproductive-age females were found to have a significantly higher NAR. As a negative appendicectomy carries significant risk, prospective studies are needed, in particular, to determine the feasibility, cost-effectiveness, and benefit of preoperative CT scanning in a resource-limited setting, especially in reproductive-age females.

Conflict of interest

The authors declare no conflict of interest.

Funding source

No funding source to be declared.

Ethical approval

Ethical approval was obtained from the Wits Human Research Ethics Committee (Medical):M180620 to conduct the study.

ORCID

S J Grossberg https://orcid.org/0000-0002-2648-8783

JD Brener https://orcid.org/0000-0002-3116-5460

EL Bizos https://orcid.org/0000-0001-8148-2678

GP Candy https://orcid.org/0000-0002-5449-1277

DB Bizos https://orcid.org/0000-0003-2506-1662

 

REFERENCES

1. Di Saverio S, Podda M, De Simone B, et al. Diagnosis and treatment of acute appendicitis - 2020 update of the WSES Jerusalem guidelines. World J Emerg Surg. 2020;15(1):27. https://doi.org/10.1186/s13017-020-00306-3.         [ Links ]

2. Seetahal SA, Bolorunduro OB, Sookdeo TC, et al. Negative appendectomy - a 10-year review of a nationally representative sample. Am J Surg. 2011;201(4):433-7. https://doi.org/10.1016/j.amjsurg.2010.10.009.         [ Links ]

3. Shrestha R, Ranabhat S, Tiwari M. Histopathologic analysis of appendectomy specimens. Journal of Pathology of Nepal. 2012;2(3):215-9. https://doi.org/10.3126/jpn.v2i3.6025.         [ Links ]

4. Jeon BG. Predictive factors and outcomes of negative appendectomy. Am J Surg. 2017;213(4):7318. https://doi.org/10.1016/j.amjsurg.2016.05.020.         [ Links ]

5. Alnaser MK, Hassan QA, Hindosh LN. Effectiveness of conservative management of uncomplicated acute appendicitis - a single hospital based prospective study. Int J Surg Open. 2018;10:1-4. https://doi.org/10.1016/jijso.2017.11.007.         [ Links ]

6. Raja AS, Wright C, Sodickson AD, et al. Negative appendectomy rate in the era of CT - an 18-year perspective. Radiology. 2010;256(2):460-5. https://doi.org/10.1148/radiol.10091570.         [ Links ]

7. Tseng J, Cohen T, Melo N, Alban RF. Imaging utilization affects negative appendectomy rates in appendicitis - an ACS-NSQIP study. Am J Surg. 2019;217(6):1094-8. https://doi.org/10.1016/j.amjsurg.2018.12.072.         [ Links ]

8. Colson M, Skinner KA, Dunnington G. High negative appendectomy rates are no longer acceptable. Am J Surg. 1997;174(6):723-7. https://doi.org/10.1016/S0002-9610(97)00183-9.         [ Links ]

9. Flum DR, Koepsell T. The clinical and economic correlates of misdiagnosed appendicitis - nationwide analysis. Arch Surg. 2002;137(7):799-804. https://doi.org/10.1001/archsurg.137.7.799.         [ Links ]

10. Giljaca V, Nadarevic T, Poropat G, Nadarevic VS, Stimac D. Diagnostic accuracy of abdominal ultrasound for diagnosis of acute appendicitis: systematic review and meta-analysis. World J Surg. 2017;41(3):693-700. https://doi.org/10.1007/s00268-016-3792-7.         [ Links ]

11. Rud B, Vejborg TS, Rappeport ED, Reitsma JB, Wille-Jergensen P. Computed tomography for diagnosis of acute appendicitis in adults. Cochrane Database Syst Rev. 2019;2019(11):CD009977. https://doi.org/10.1002/14651858.CD009977.pub2.         [ Links ]

12. Coursey CA, Nelson RC, Patel MB, et al. Making the diagnosis of acute appendicitis - do more preoperative CT scans mean fewer negative appendectomies? A 10-year study. Radiology. 2010;254(2):460-8. https://doi.org/10.1148/radiol.0908229812.         [ Links ]

13. Silber W. Appendicectomy - a survey of 200 cases. S Afr Med J. 1953;27(9):187-90. PMID: 13076641.         [ Links ]

14. Spitz L. Acute appendicitis. An analysis of six hundred and sixty-six appendicectomies in adults 1959-1968. S Afr J Surg. 1969;7(3):129-37. PMID: 5369455.         [ Links ]

15. Nel CJ, Theron EJ. Appendisitis by die Swart bevolking [Appendicitis in the Black population]. S Afr Med J. 1979;55(23):939-41. PMID: 472933.         [ Links ]

16. Fulton J, Lazarus C. Acute appendicitis among black South Africans. S Afr J Surg. 1995;33(4):165-6. PMID: 8677468.         [ Links ]

17. Levy RD, Degiannis E, Kantarovsky A, et al. Audit of acute appendicitis in a black South African population. S Afr J Surg. 1997;35(4):198-202. PMID: 9540398.         [ Links ]

18. Muthuphei MN, Morwamoche P. The surgical pathology of the appendix in South African blacks. Cent Afr J Med. 1998;44(1):9-11. PMID: 9631107.         [ Links ]

19. Madiba TE, Haffejee AA, Mbete DL, Chaithram H, John J. Appendicitis among African patients at King Edward VIII Hospital, Durban, South Africa: - a review. East Afr Med J. 1998;75(2):81-4. PMID: 9640828.         [ Links ]

20. Rogers AD, Hampton MI, Bunting M, Atherstone AK. Audit of appendicectomies at Frere Hospital, Eastern Cape. S Afr J Surg. 2008;46(3):74-7. PMID: 18807302.         [ Links ]

21. Chamisa I. A clinicopathological review of 324 appendices removed for acute appendicitis in Durban, South Africa: a retrospective analysis. Ann R Coll Surg Engl. 2009;91(8):688-92. https://doi.org/10.1308/003588409X12486167521677.         [ Links ]

22. Kong V, Handley J, Furlong H, Allorto N, Clarke DL. Tracking the course of complicated appendicitis through a regional hospital and developing a feedback mechanism to referring district hospitals. In: Raemaekers M. SAJS 39th Annual Meeting of the Surgical Research Society of Southern Africa: 23-24 June 2011. S Afr J Surg. 2012 Oct:1-34. Available from: http://sajs.redbricklibrary.com/index.php/sajs//article/view/1770.         [ Links ]

23. Kong VY, Bulajic B, Allorto NL, Handley J, Clarke DL. Acute appendicitis in a developing country. World J Surg. 2012;36(9):2068-73. https://doi.org/10.1007/s00268-012-1626-9.         [ Links ]

24. Nshuti R, Kruger D, Luvhengo TE. Clinical presentation of acute appendicitis in adults at the Chris Hani Baragwanath academic hospital. Int J Emerg Med. 2014;7(1):12. https://doi.org/10.1186/1865-1380-7-12.         [ Links ]

25. Yang E, Cook C, Kahn D. Acute appendicitis in the public and private sectors in Cape Town, South Africa. World J Surg. 2015;39(7):1700-7. https://doi.org/10.1007/s00268-015-3002-z.         [ Links ]

26. Jolayemi OO, Moodley NB, Kong VY, et al. The usefulness of routine histological examination of appendicectomy specimens in a South African tertiary centre. S Afr Med J. 2018;108(4):342-6. https://doi.org/10.7196/SAMJ.2018.v108i4.12759.         [ Links ]

27. Yang E, Kahn D, Cook C. Acute appendicitis in South Africa: a systematic review. S Afr J Surg. 2015;53(3&4):1-8. PMID: 28240480.         [ Links ]

28. Arthur T, Gartrell R, Manoharan B, Parker D; QUEST Collaboration. Emergency appendicectomy in Australia: findings from a multicentre, prospective study. ANZ J Surg. 2017;87(9):656-60. https://doi.org/10.1111/ans.14088.         [ Links ]

29. Van Rossem CC, Bolmers MD, Schreinemacher MH, Van Geloven AA, Bemelman WA; Snapshot Appendicitis Collaborative Study Group. Prospective nationwide outcome audit of surgery for suspected acute appendicitis Br J Surg. 2016;103(1):144-51. https://doi.org/10.1002/bjs.9964.         [ Links ]

30. National Surgical Research Collaborative. Multicentre observational study of performance variation in provision and outcome of emergency appendicectomy. Br J Surg. 2013;100(9):1240-52. https://doi.org/10.1002/bjs.9201.         [ Links ]

31. SCOAP Collaborative. Cuschieri J, Florence M, Flum DR, et al. Negative appendectomy and imaging accuracy in the Washington state surgical care and outcomes assessment program. Ann Surg. 2008;248:557-63. https://doi.org/10.1097/SLA.0b013e318187aeca.         [ Links ]

32. Drake FT, Florence MG, Johnson MG, et al. Progress in the diagnosis of appendicitis: a report from Washington State's Surgical Care and Outcomes Assessment Program. Ann Surg. 2012;256(4):586-94. https://doi.org/10.1097/SLA.0b013e31826a9602.         [ Links ]

33. Musunuru S, Chen H, Rikkers LF, Weber SM. Computed tomography in the diagnosis of acute appendicitis: definitive or detrimental? J Gastrointest Surg. 2007;11(11):1417-22. https://doi.org/10.1007/s11605-007-0268-y.         [ Links ]

34. Perez J, Barone JE, Wilbanks TO, Jorgensson D, Corvo PR. Liberal use of computed tomography scanning does not improve diagnostic accuracy in appendicitis. Am J Surg. 2003;185(3):194-7. https://doi.org/10.1016/S0002-9610(02)01364-8.         [ Links ]

35. Frei SP, Bond WF, Bazuro RK, et al. Appendicitis outcomes with increasing computed tomographic scanning. Am J Emerg Med. 2008;26(1):39-44. https://doi.org/10.1016/j.ajem.2007.06.027.         [ Links ]

36. Vadeboncoeur TF, Heister RR, Behling CA, Guss DA. Impact of helical computed tomography on the rate of negative appendicitis. Am J Emerg Med. 2006;24(1):43-7. PMID: 16338508.         [ Links ]

37. Flum DR, McClure TD, Morris A, Koepsell T. Misdiagnosis of appendicitis and the use of diagnostic imaging. J Am Coll Surg. 2005;201(6):933-9. https://doi.org/10.1016/j.jamcollsurg.2005.04.039.         [ Links ]

38. Antevil J, Rivera L, Langenberg B, Brown CV. The influence of age and gender on the utility of computed tomography to diagnose acute appendicitis. Am Surg. 2004;70(10):850-3. PMID: 15529835.         [ Links ]

39. McGory ML, Zingmond DS, Nanayakkara D, Maggard MA, Ko CY. Negative appendectomy rate - influence of CT scans. Am Surg. 2005;71(10):803-8. PMID: 16468523.         [ Links ]

40. Morse BC, Roettger RH, Kalbaugh CA, Blackhurst DW, Hines WB Jr. Abdominal CT scanning in reproductive-age women with right lower quadrant abdominal pain: does its use reduce negative appendectomy rates and healthcare costs? Am Surg. 2007;73(6):580-4. PMID: 17658095.         [ Links ]

 

 

Correspondence:
SJ Grossberg
Email: saulgrossberg@gmail.com

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons