Review of the Quality of Recorded Crime Statistics 2023

This chapter provides details of an assessment of the quality of Recorded Crime data as part of this Quality Review. The CSO has carried out a series of checks on sample data for different years since 2011. The results in Table 3.1 below indicate a sustained level of improvement over time in these key metrics of PULSE data quality.

Summary results of quality assessments

Table 3.1 below outlines the results of the data quality checks that CSO has carried out on a sample of PULSE data for each year as part of the various quality assessments since 2011. The text following this table provides more detail of these quality checks.

Detailed description of quality checks

The following section describes each of the quality checks listed in Table 3.1.

      1. Correct classification of crime incident type

A crime incident is classified by offence type (e.g., theft, burglary) when it is entered on PULSE. All classifications are subject to review and supervision, and a reclassification can be applied where the original classification is deemed to be incorrect due to error or new evidence.

The CSO selected a random sample of 50 incidents reported in Q1 2023 in each of six selected crime incident types on PULSE (i.e 300 in total). The CSO considered the information contained in descriptive data fields to assess whether the incident classification was appropriate based on the information reviewed, as in previous CSO reviews. Of the 300 incidents reviewed, 295 (98%) were considered to have been classified correctly based on the descriptive data fields examined. The remaining five incidents (2%) were either considered to have been misclassified or there was insufficient detail recorded in the description to support the classification. This shows further improvement on previous CSO reviews where an estimated 97% of records in 2021, 96% in 2017, 95% of records in 2015 and 93% in 2011 were assessed as having been classified correctly. See Table 3.1 and Figure 3.1.

      2. Classification of crimes in ‘non-crime’ incident categories

As well as recording criminal offences, PULSE is used to record other Garda activity where a criminal offence has not occurred (property reported lost or found, traffic checkpoints, etc.). The inappropriate use of non-crime incident categories when recording criminal offences was highlighted as a data quality issue in the Garda Inspectorate’s Crime Investigation report. Non-crime incident records are not counted in Recorded Crime statistics.

The CSO sampled 200 non-crime incident records of three types reported between January and March of 2023 (Q1 2023): ‘Attention and Complaints’, ‘Property Lost’ and ‘Domestic Dispute – no offence disclosed’ and examined the narrative and related data fields to assess whether the classification of the incident as a non-crime had been accounted for, either in the incident’s description or by explicit statement that no offence had taken place. The CSO considered that all 200 non-crime incident records examined were correctly classified as non-crime based on the narrative field. This compares with figures of 99% in 2021, 98% in 2017, 97% in 2015 and 92% in 2011. See Table 3.1 and Figure 3.2.

      3. Crimes marked ‘invalid’ on PULSE

Crime incidents can be marked invalid on PULSE if it is determined that a crime did not occur, or if the record is a duplicate or created in error. Invalidated PULSE crime incidents are not counted in Recorded Crime statistics. The Garda Inspectorate’s Crime Investigation report had outlined the incorrect invalidation of incidents as a data quality concern.

AGS has introduced control measures to prevent inappropriate invalidation, such as increased controls over and tracking of changes to the incident narrative or the status of the incident (reclassified, invalidated etc.). Reports are compiled monthly which details the number of incidents invalidated during the month by either the Garda Information Service Centre (GISC) reviewers or Garda Members.  This report is then actioned to review the merit of incidents marked invalid by Garda Members.  Incidents found to have been marked invalid incorrectly are revalidated by the Data Quality section and returned to the Garda Member for update. 

CSO sampled 100 invalidated crime incident records (of around 1,100 which would otherwise have been counted) reported in Q1 2023 and examined the text of the narrative and related data fields to assess whether the reason for invalidation had been recorded. The CSO considered that 94% of the sampled invalidated records had the reason for invalidation recorded. This compares with findings of 92% in 2021, 88% in 2017, 79% in 2015 and 77% in 2011. See Figure Table 3.1 and Figure 3.3.

      4. Adherence to current crime counting rules

The PULSE recording guide outlines the rules around three important crime counting principles which are to be applied by AGS when recording crime incidents. Incorrect application of these rules can impact on the coverage of the data used to compile Recorded Crime statistics. The counting rules are:

• Primary offence rule: “Where two or more criminal offences are disclosed in a single episode it is the primary criminal offence that is counted”
• One offence counts per victim
• Continuous series of offences involving same victim and same offender: - “A continuous series of offences against the same victim involving the same offender counts as one offence”

In practice, it is the primary offence flag that is frequently used when recording incidents on PULSE to try to ensure that multiple offences are counted correctly, by marking those which are not intended to be counted as non-primary in any of the above scenarios. 

The CSO extracted a random sample of 100 non-primary incidents on PULSE reported in 2020, along with all associated primary or non-primary incidents, and examined each set of incidents together to consider compliance with the current counting rules. CSO consider that 99 of the 100 sampled sets of incidents complied properly with the counting rules. The findings of this assessment indicate there is consistent application of the crime counting rules.

Further, CSO extracted all PULSE records reported in 2020 in the categories of sexual violence and fraud where a single victim was linked to more than one crime incident and considered the application of the ‘Continuous series of offences involving same victim and same offender’ rule based on available data including the date occurred, date reported, suspected offender ID and the narrative data field. CSO estimate that for 12 incidents of sexual violence in Q1 2023 (2% of such incidents) and 24 incidents of fraud involving a recorded victim (1% of such incidents) this rule was incorrectly applied on PULSE. The underlying crime incidents were however recorded on PULSE. See Table 3.1 and Figure 3.4.

      5. Recorded Crime Detection series

The CSO Recorded Crime Detections series outlines the proportion of incidents reported in the Recorded Crime series which have been marked as detected. As such, quality considerations outlined in respect of the coverage of the Recorded Crime series must also be considered in respect of detections.

PULSE Upgrade 7.3, released on 25 February 2018, introduced new data governance controls in respect of recording detections. Following the PULSE update, the recording of a sanction such as charge, summons or caution against a suspect should automatically update other data fields on PULSE, all of which are used by the CSO for statistical purposes:

• detection status of the crime incident
• detection date of the crime incident (i.e., corresponding to date sanction was applied)
• classification of the suspect to suspected offender
• detection status (which indicates the type of sanction) of the suspected offender

CSO examined PULSE data relating to charges and summons issued in respect of incidents reported in Recorded Crime statistics in 2020, and checked the dependent data fields of the crime incident and suspected offender for coherence, with the following findings:

• In practically 100% of cases, a valid charge or summons corresponded with an incident being marked as detected or resulted in proceedings.
• In more than 99% of cases, the year of the first valid charge or summons associated with a PULSE crime incident corresponded with detection date year. Note that in some cases an alternative sanction (i.e., caution) may have been applied earlier, however CSO were unable to test this with available data, and so the accuracy may be higher than reported here.
• In more than 99% of cases, a valid charge or summons corresponded with the presence of a valid suspected offender record.
• In 99% of cases, a valid charge or summons corresponded with the detection status of a valid suspected offender record indicating that proceedings had commenced.

These findings highlight the impact of stricter data governance controls implemented in PULSE 7.3 on data quality, ensuring greater coherence of data in respect of detected incidents and suspected offenders across the PULSE system and better data outputs for statistical purposes. See Table 3.1 and Figure 3.5.

      6. Timely recording of PULSE crime incident records

When a criminal offence is reported to AGS it should be recorded on PULSE as soon as possible. Minimising the delay when recording offences ensures that critical information is available in as close to real-time as possible and enhances the integrity of the record relative to the report.

CSO analysed the date reported data field for PULSE crime incidents of selected types[1] for each of the reference periods of the CSO quality reviews (Q1 2023, 2011, 2015, 2017, 2020) relative to the system-generated date created data field for the record. The proportion of crime incidents which were recorded on PULSE either on the same day they were first reported, or the next day increased from 89% of incidents in the selected incident types in 2011, to 91% of incidents in 2015 and 2017, to 93% of incidents in both 2020 and in Q1 2023. The proportion of incidents where the gap was two weeks or more halved, from 4% to 2%, over the same period.

[1] Homicides (excl. dangerous driving leading to death), sexual violence, assault, robbery, burglary, theft, drug offences, criminal damage, public order offences. See Table 3.1 and Figure 3.1.

      7. Recorded of crime victims

The CSO Recorded Crime Victims series analyses the set of victims linked to incidents reported in the Recorded Crime series. As such, quality considerations outlined in respect of the coverage of the Recorded Crime series must also be considered in respect of victims.

Where a person is the victim of a criminal offence, details of the person are recorded on PULSE and the person is classified as the ‘injured party’. Not all criminal offences have a victim who is a person, but crimes such as homicide, physical and sexual violence, robbery from the person and theft from the person would have an identified victim in almost all cases. Details such as the sex and date of birth of the victim are routinely recorded on PULSE and CSO uses this data to compile annual statistics in respect of victims of serious crimes such as homicide, sexual violence and assaults classified by sex and age group.

The accuracy of the profiles provided in Recorded Crime statistics is dependent on two main factors: the extent to which an ‘injured party’ has been recorded; and the completeness and accuracy of the data fields relating to sex and date of birth. CSO examined the extent to which an ‘injured party’ had been recorded in respect of the set of crime incidents examined in the Recorded Crime Victims statistical tables of for Q1 2023, as well as the completeness of the sex and date of birth data field, with the following findings:

• A valid person profile classified as ‘injured party’ with valid sex and date of birth was linked to all (100%) of homicide incidents reported in Q1 2023
• A valid person profile classified as ‘injured party’ with valid sex and date of birth was linked to 99% of sexual violence incidents reported in Q1 2023.
• A valid person profile classified as ‘injured party’ with valid sex and date of birth was linked 98% of incidents classified as assaults and relating offences.

Note that not all incident categories in this group must necessarily have an injured party, and an update was issued on PULSE during 2020 which corrected a known exclusion and improved coverage. See Table 3.1.

      8. Crimes that are reported but are not recorded on PULSE

In 2014, the Garda Inspectorate’s Crime Investigation report highlighted the issue of crimes that are reported to AGS but are not recorded on PULSE as being of concern. In previous CSO quality reviews, the CSO sampled records from the Computer Aided Dispatch (CAD) and other incident logging registers to provide estimates for the proportion of reports which appeared to be crimes (based on the recorded descriptions) but were not represented on PULSE as crime incidents. Based on these checks, the CSO estimated that the non-recording of reported crimes had fallen from 18% in 2011, to 12% in 2015 and to 7% in 2017.

CAD is used by AGS for all logging of initial reports of incidents made by members of the public by telephone, including via 999 emergency calls. The launch of a new CAD 2 system by AGS has meant that the CAD and PULSE systems are now integrated such that a PULSE incident is automatically created when a crime-related CAD incident is closed. This system was introduced in part as a quality assurance mechanism to prevent a scenario where a crime reported by a member of the public was not recorded on PULSE.

In June 2021, the Garda Commissioner outlined to the Policing Authority details relating to the inappropriate cancellation of CAD incidents. The premature or improper cancellation of calls on the CAD system may have meant that records relating to crimes, which were reported to AGS, were not created on the PULSE system, and were therefore not counted in Recorded Crime statistics.

AGS carried out an investigation on the most serious high-risk crimes of the cancelled 999 incidents. These mainly related to Domestic Violence, Sexual Assault, Health and Missing Persons, which covered the cancellation of about 6,000 incidents dating back to 2019. An incident can be validly cancelled for a number of reasons such as, for example, multiple calls being received for the same incident. Of the incidents which should not have been cancelled based on the AGS investigation of the issue, 141 would have resulted in a criminal incident being recorded on PULSE. All 141 missing incidents were subsequently created on PULSE. In volume terms, this had a negligible effect on published Crime Statistics for the period in question.

      9. Recorded Crime Suspected Offenders series

The CSO Recorded Crime Suspected Offenders series analyses the set of suspected offenders linked to incidents reported in the Recorded Crime series. The accuracy and reliability of suspected offender data is closely tied to the accuracy of detections data, due to the data governance controls implemented in PULSE 7.3. Please refer to the section relating to the Recorded Crime Detections series above for analysis.

Conclusion

Based on the checks outlined above, it is clear that the quality of PULSE data has been improving steadily over time and is currently at a high level. This reflects the strong data controls implemented by AGS over time on PULSE data. In particular, considerable improvement has been made in adherence to crime counting rules and this is helped by the system of checks on this issue implemented by AGS progressively over time.