Cyber Secure Behavior Lab

Developing a secure programming concept inventory

secure programming concept inventory

The goal of this project is to develop a concept inventory for secure programming. A concept inventory is an assessment tool designed to measure an individual’s understanding of concepts in a specific knowledge domain.  For this project, the knowledge domain is that of secure, or defensive, programming. The rapid pace of innovation in computer science means that practitioners are needed who are able to write safe computer code, design systems in new ways to respond to new security needs and respond to new threats. This requires that practitioners have a clear understanding of the foundational concepts in secure programming to serve as a basis for building new knowledge and responding to new challenges.

However, there is a lack of tools to reliably measure students’ understanding of foundational concepts. We propose the development of a Secure Programming Concept Inventory (SPCI) designed to measure an individual’s understanding of foundational secure programming concepts. To do this, we will draw on prior work to identify the foundational, knowledge critical concepts in the domain of secure programming. Next, we will identify hard topics and common misconceptions held by students, especially those related to the foundational and critical concepts. We will develop a pool of items that specifically target difficult concepts and misconceptions. This pool will enable us to assess how well students understand and can work with these concepts. Finally, we will test and refine the pool of items to develop a validated secure programming concept inventory. This concept inventory will be developed in three parts each designed to answer specific research questions and produce deliverables. 

These are:

  • Part 1 – Identifying misconceptions: The first part of the project will investigate what the common misconceptions are in secure programming and which concepts students find difficult. We will use interviews with students and instructors to answer this question. The deliverable for this part will be a set of concepts that students commonly misunderstand or find difficult.
  • Part 2 – Developing the item pool: The second part of the project will develop scale items that specifically target learning, difficult concepts, and misconceptions in secure programming. The deliverable for this part will be a large pool of 150-200 multiple choice items addressing foundational concepts in secure programming.
  • Part 3 – Testing and refining items: The final part of the project will investigate which items best target conceptual understanding. This will be done by administering the inventory items developed in part 2 to students and using performance indicators to refine or discard weak items. The deliverable for this part will be the validated concept inventory.

 

This project will advance the understanding of how students learn secure programming by identifying major misconceptions and the sources of these misconceptions, and by providing a reliable and validated instrument to aid in assessment and curricular design. This project is a collaboration among researchers at Purdue University, California State University Sacramento, University of California Davis, and California Polytechnic State University San Luis Obispo.

 

 

Developing a Secure Programming Concept Inventory. Sponsoring Organization: National Security Agency. Role: PI. Total Award: $299,931. Award: $299,931. 09/13/2017 – 09/12/2019.