The Internet of Things ("IoT") is an ecosystem where connected devices interact with other devices and services. In this ecosystem, IoT products combine hardware, software and network capabilities to collect data, analyse them and perform tasks.
This creates a large attack surface, with vulnerabilities causing risks to systems, data, and users (privacy, safety). Due to the large number of IoT systems in the world, a vulnerability in one product can affect millions of systems at the same time. And because of their nature, most IoT products are an easy target for attackers.
To limit the impact of insecure IoT on our society, multiple cyber security regulations are now in place. These regulations require IoT manufacturers to manage cyber risks during the product development phase and after product release. Manufacturers must keep track of vulnerabilities, assess their risks on products, users and data, and fix them (usually with a patch).
For that purpose, most regulations require IoT manufacturers to implement a public Vulnerability Disclosure Policy ("VDP") to receive vulnerability reports from external stakeholders. To do so, manufacturers can follow international standards such as ETSI EN 303 645 (provision 5.2-1), ETSI TR 103 838 or ISO 29147.
A public VDP is a good tool to improve product security. This is also a requirement of most IoT cyber security regulations. However, VDP adoption in consumer IoT is still limited as shown by the IoT Security Foundation and CopperHorse Ltd in their yearly reports.
We decided to explore the accessibility and usability of VDP in consumer IoT, a regulatory requirement for IoT manufacturers. Making VDP accessible and usable is not only a compliance requirement, it is necessary to encourage the submission of vulnerability reports.
We discovered that many public VDP have issues regarding accessibility or usability. We also found that international cyber security standards do not include any accessibility requirements.
In this study, we evaluate the accessibility and usability of VDP in consumer IoT by looking at several metrics. We highlight various issues that limit the effectiveness of VDP and that could cause issues to manufacturers with their regulatory compliance. Finally, we propose a list of recommendations to make IoT VDP more accessible. This will improve product security and help manufacturers comply with their regulatory requirements more easily.
List of issues
- Issue 1. VDP is not directly accessible
- Issue 2. Inconsistent accessibility across products or markets
- Issue 3. Public VDP does not cover products
- Issue 4. Invalid security.txt
- Issue 5. VDP accessible in an unusual way
- Issue 6. Dedicated product security website not directly accessible
- Issue 7. Vulnerability reporting is private
- Issue 8. Usability issues in VDP webpage
- Issue 9. Poor usability of reporting template
- Issue 10. Products compliant with EN 303 645 with no VDP
List of recommendations
- Recommendation 1. Implement a global VDP strategy ("one VDP")
- Recommendation 2. Have a link to the VDP in the footer menu
- Recommendation 3. Keep the VDP open and public
- Recommendation 4. Have a valid security.txt
- Recommendation 5. Make it easy to reach the point of contact
- Recommendation 6. Prefer a webform for vulnerability reporting
- Recommendation 7. Integrate VDP accessibility in standards and regulations
- Recommendation 8. Follow web accessibility best practices
Scope and Methodology
The scope of this study focuses on the accessibility of the vulnerability disclosure policy and its usability in various consumer IoT products. The list of products was chosen randomly among renowned brands, products we own, products we want to buy, products with a cyber security label and some of their competitors.
We consider all variants of the VDP such as coordinated disclosure, responsible disclosure, etc.
Sectors outside IoT products are out of scope even though the results and our recommendations are directly applicable to any domain (IT, Cloud, infrastructure, OT, etc.).
Accessing public VDP
We tried accessing existing public VDP as follows:
- Open webpage of IoT product or its manufacturer
- Look for public VDP link in footer
- Look for public VDP link in header menu, submenus, in the product page, in the contact page
- Look for a valid security.txt in
- Look for keywords in an external search engine
- Look for reporting means (webform, email address)
- Evaluate VDP accessibility scoring
We mainly relied on manual navigation to discover the public VDP webpage. When we struggled to find a link, we searched for keywords such as:
- disclos* (disclose, disclosure)
VDP accessibility scoring
We looked at the following metrics to score the accessibility of public VDP. The VDP accessibility scoring depends on the sum of these metrics (over 10 points).
The following table details the different metrics.
Accessibility of the VDP from the frontpage
|Link in the footer menu
|Link in a header sub-menu
|Link in another part of the website
|VDP not accessible directly, it requires an external search engine
Number of clicks to access the VDP from the frontpage
|1 or 2 clicks
|3 or 4 clicks
|5 clicks or more
|VDP not found using navigation
Number of clicks to submit the report from the public VDP webpage
|3 clicks or more
Bonus point: presence of a valid security.txt
|Security.txt is compliant with RFC 9116
VDP accessibility score
The VDP accessibility score is either "great", "good", "limited" or "poor" according to the sum of all metrics:
- 9 or 10: 🌞 Great accessibility
- 7 or 8: 🌤️ Good accessibility
- 5 or 6: 🌧️ Limited accessibility
- 1 to 4: ⛈️ Poor accessibility
Evaluate your VDP accessibility score
VDP accessibility in standards and regulations
We looked at VDP accessibility requirements in IoT cyber security standards and regulations. Only 2 regulations require making the VDP accessible: the EU Cyber Resilience Act and the UK PSTI (with implicit requirements that could be interpreted as mandatory).
ETSI EN 303 645
The first international standard to establish a cyber security baseline for consumer IoT requires a VDP with a minimum level of information (provision 5-2.1). However, it has no requirement to make this VDP accessible.
NIST IR 8259B
NIST mandates non-technical requirements supporting their core baseline for IoT cyber security. They establish requirements for "documentation" including the publication of a VDP with a point of contact. However, there is no mention of making the VDP accessible either.
Regulatory requirements in the United Kingdom
In the UK, the PSTI Regulations 2023 requires manufacturers (and distributors) of consumer IoT to make a point of contact accessible to report vulnerabilities in Schedule 2, Art. 2(3).
These accessibility requirements only concern the point of contact but they should easily apply to the VDP itself.
Schedule 2, Art. 2(3) of the UK PSTI Regulations
Information on how to report security issues
(3) The information in sub-paragraph (2) must be accessible, clear and transparent, and must be made available to [a person] P—
(a) without prior request for such information being made;
(b) in English;
(c) free of charge; and
(d) without requesting the provision of P's personal information.
Regulatory requirements in the European Union
The Radio Equipment Directive Delegated Act ("RED DA") for Cyber Security is a technical regulation and it does not require a vulnerability disclosure policy.
The Cyber Resilience Act is an upcoming regulation for products with digital elements. Article 10(10) in the final text requires IoT manufacturers to make accessible their requirements in Annex II which include a VDP.
Annex II of the EU Cyber Resilience Act
Information and instruction to the user
2. the single point of contact where information about vulnerabilities of the product with digital elements can be reported and received, and where the manufacturer's policy on coordinated vulnerability disclosure can be found
VDP accessibility issues
We discovered several issues that make reporting vulnerabilities difficult. These issues appear at multiple IoT manufacturers, no matter their sector, their size or their geography. Moreover, some of these issues may lead to early disclosure, affecting the security of IoT products, customers and users, as well as the reputation of their manufacturer.
With this study, we encourage all consumer IoT manufacturers to review the accessibility and usability of their VDP in the light of these different issues. We released a free tool to evaluate your VDP accessibility score.
Evaluate your VDP accessibility score
Issue 1. VDP is not directly accessible
When the VDP exists, it should be accessible easily, either from the product website or from the manufacturer's corporate website. Sometimes this VDP is not directly accessible because it takes too many clicks to access it or there is no direct link to it.
It is also possible to access this VDP by using a search engine. However, this is not optimal and vulnerability reporters may lose patience, going to social media to find a point of contact or by making their findings public immediately. This could create reputational risks and other internal troubles for IoT manufacturers.
The easiest solution is to make the VDP directly accessible on the product website.
Issue 2. Inconsistent accessibility across products or markets
We explored how IoT manufacturers implement their VDP across different markets and / or across different brands and subsidiaries. We discovered several interesting issues:
- Multiple VDPs for different products or brands. This can lead to a misalignment in scope or treatment of reports, especially when these products or brands share the same resources (development team, source code, user account, backend services).
- No VDP for some products or brands. This could rapidly become confusing when other products or brands have a VDP.
- Different reporting requirements for different markets. This denotes a compliance-driven approach which is not helpful for security.
- No VDP for similar products on different markets. This is not acceptable: the same products should have similar requirements across all markets.
We can imagine that these issues are related to the internal organisation of big corporations. However, we believe they could benefit from simplifying their product security with one single VDP for all markets, all products and all brands. This will make things more efficient and easier to manage.
Issue 3. Public VDP does not cover products
Some manufacturers of consumer IoT have a public VDP that does not cover their products. Their existing VDP would only cover public websites and potentially Cloud APIs.
This could quickly become an issue if product vulnerabilities are not reported or worse, if valid reports are deemed "out of scope" and simply ignored. In this case, reporters could decide to make their findings public to avoid unnecessary effort with the manufacturer.
Issue 4. Invalid security.txt
The security.txt is currently not a requirement in IoT cyber security standards. Yet, it is a great way to make a VDP more accessible:
- It is always located at the same URL
- It must point to the policy webpage and should communicate a single point of contact.
- It is a simple text file that only requires little maintenance.
However, many IoT manufacturers with an accessible VDP do not have an associated security.txt. And when they do, it is possible that their security.txt is invalid:
- The security.txt expiration date is in the past. This is the most common issue as setting up a security.txt is usually done once and content update is often overlooked. However, this is not really an issue in real-life if the URLs are still accessible.
- The security.txt is incomplete and does not contain all mandatory fields (a link to the VDP and the expiration date).
- Wrong content in the security.txt. In most cases, the security.txt contains one or more stale links. In one instance, we discovered an invalid security.txt pointing to the VDP of the CRM vendor.
Issue 5. VDP accessible in an unusual way
Ideally, product websites contain a self-explanatory link pointing to the VDP. Reporters can find this link on the frontpage (in the footer menu or in the header menu), in a contact page, or in a thematic webpage.
We discovered that some manufacturers place their VDP in very unusual locations that make it very difficult to access, negating any benefits:
- The VDP is accessible through the privacy notice webpage. This link to the "privacy notice" is usually found in the footer menu. However, it is an unexpected location for product VDP. Indeed, consumer IoT vulnerabilities go way beyond personal data.
- The VDP is located under the "legal & compliance" webpage. This is an unusual location, as it is generally a place for displaying corporate policies (supply chain, sustainability, etc.), not product ones.
- VDP is hidden in a blog post, in a support page or in a community forum post. This is not acceptable as nobody would actively search the VDP in these locations (sure, we did but only because we are doing this study).
We believe that these manufacturers follow a compliance-driven approach to security. We can only imagine that they follow the same approach for their product development, which is not optimal (to stay polite). We encourage them to improve the accessibility of their VDP by following our recommendations.
Issue 6. Dedicated product security website not directly accessible
Some IoT manufacturers have a dedicated product security website containing their VDP. This is usually the website of the Product Security Incident Response Team (PSIRT) which is responsible for keeping products secure after release. These websites are usually hosted on their own domain or on a subdomain of the corporate website.
And yet, in several instance it is impossible to reach the dedicated product security website from the product/manufacturer website. It is only possible to discover their existence using an external search engine.
These manufacturers have a very mature security posture in the IoT ecosystem. They should lead by example and make their dedicated product security website more accessible.
Issue 7. Vulnerability reporting is private
To comply with regulations, vulnerability reporting must be accessible publicly. This means that the point of contact to submit a report must be open, without a user account.
We discovered issues that go against these regulatory requirements:
- Several manufacturers require a login to submit vulnerability reports using an external platform. This means that security researchers must create an account on a third-party service and accept the associated terms and conditions (which can be unfair to reporters). Note that it is perfectly acceptable to use an external platform for VDP reporting as it can be difficult to implement all requirements internally due to the lack of processes, resources or skills.
- Manufacturers with their own VDP reporting capabilities require a user account to submit security issues. Even though it may be a requirement for their bug bounty, public VDP reporting must remain public as security issues affect products on the market.
These IoT manufacturers must understand that their point of contact must be accessible easily as required by IoT cyber security regulations. If they do not align with this requirement, they may be facing important penalties in the coming months.
Ring requires a login to report issues via Amazon dedicated page (hosted on HackerOne)
Apple requires a user account to submit a vulnerability report
Issue 8. Usability issues in VDP webpage
When the VDP reporting page is easily accessible, it is possible that user experience creates issues to reporters. We discovered some usability issues that are easy to fix:
- A reporting webform opening a new link to the VDP. When clicking on this link, a new page opens in the same browser tab. This webpage has no link to go back to the webform. The only solution is to use the "Previous" button on the browser.
- The impossibility to report a vulnerability on all browsers. On one occasion, a manufacturer uses a text box with a button to access their VDP. This button becomes hidden on mobile due to overflow. On another occasion, only Chrome-based browsers display the relevant reporting procedure
- An unnecessary complicated workflow to submit a report. This can discourage most reporters.
Eufy VDP is accessible from the reporting webform but there is no link to go back
Huawei VDP is not accessible on mobile. The "Learn More" button disappears the due to overflow
Samsung reporting is unnecessary complicated and is broken on Firefox
Issue 9. Poor usability of reporting template
A VDP defines the rules for reporting vulnerabilities. When the VDP mandates a reporting template, reports must contain specific information. In turn, manufacturers who receive standardised reports can better identify issues and fix root causes.
The VDP usually communicates this reporting template with an outline or with specific fields in a webform. We discovered that some manufacturers require reporters to submit a binary file. We believe this causes some issues:
- Reporters will have to download a binary file, install the appropriate editing tool and send their final report. This is cumbersome and some reporters may decide to use their own template, or not to disclose at all.
- Binary file formats like .docx or .pdf could potentially leak metadata when the reporter wishes to remain anonymous. This can cause issues in geographies that do not protect vulnerability reporters from legal repercussions.
- A reporting template using a spreadsheet (usually an Excel file) will limit the number of characters per cell. This may require some editing (new lines, new cells) that may break any automation in place on the receiving side.
- Binary file formats could be used for introducing malware at reporters and at manufacturers. This requires additional security requirements.
TP-Link reporting template is an Excel spreadsheet with a character limit per cell
Samsung reporting template shows is leaking internal data due to track changes enabled. This could cause further security risks since the editor is "Administrator"
Issue 10. Products compliant with EN 303 645 with no VDP
In this study, we looked at several products that advertise their compliance with ETSI EN 303 645. They usually display a label issued by a governmental agency or by a private test lab.
As a reminder, a VDP with a single point of contact and clear reporting timelines is a mandatory requirement in EN 303 645 (provision 5.2-1).
We found an issue with a government-issued label based on EN 303 645: Tietoturvamerkki. We could not find any VDP for labelled product made by Finnish manufacturers, despite manufacturers reporting their compliance. These manufacturers only have a point of contact. This raises several questions on the credibility of such labels.
For example, Polar declared a publicly available VDP to Traficom, who oversees the label. They only have an email address in a hidden support page, with no timeline (even though it is a requirement of EN 303 645). We contacted Polar and Traficom the 16th of November 2023 with our enquiry. We are still waiting for an answer.
Issues with EN 303 645 compliance issues by the Finnish label (Tietoturvamerkki)
We propose several recommendations to optimize the accessibility and usability of VDP in consumer IoT. By following them, manufacturers will improve the accessibility and usability of their VDP and its effectiveness.
One of the biggest challenge IoT manufacturers are facing today is the regulatory overhead. A compliance-based approach can only cater to individual regulations, which will lead to potential gaps and issues. For instance, the EU RED DA does not require a VDP.
We recommend designing and implementing a global VDP strategy for consumer IoT products ("one VDP"). This "one VDP" should be the unique reference for external vulnerability reporters and for internal stakeholders: all products, all brands and all markets should have the same rules.
When designing their "one VDP", manufacturers must integrate all relevant regulatory requirements as well as internal constraints (for example, dedicated product security teams per brand or per market).
The most common way for accessing a VDP is in the footer menu. This link would generally point to the VDP or to a dedicated product security website.
We recommend placing a link to the VDP in the footer with a self-explaining text such as "report a vulnerability" or "product security". Manufacturers should not place this link under any non-intuitive page such as the privacy notice, legal & compliance, support or FAQ.
The VDP and the point of contact must be clear and accessible freely. This is particularly true to comply with regulations in the UK and in the EU.
We recommend all manufacturers to keep their VDP open and public. This means removing login requirements and any other pre-requisite. This recommendation also applies to third-party platforms used for reporting.
Manufacturers can place their VDP on any URL. This means that vulnerability reporters must navigate on a website and look for links to the VDP before submitting their findings. This study has highlighted several issues associated to this approach.
Manufacturers should formalise their VDP information in a
security.txt file, and place it in the
.well-known/ directory of their
product website. This will improve VDP accessibility by giving direct
access to the relevant VDP information.
At cetome, we support the deployment of security.txt on products websites and on the corporate website. It brings immediate benefits with a low level of effort for implementation and maintenance.
The point of contact should remain easy to reach, within an acceptable number of click and with no additional requirements. For example, reporters should not have to create an account.
A webform is the easiest way to implement a template and automate its treatment. It is easier to manage than emails with PGP and it could improve security by removing file attachments. Moreover, manufacturers who use binary files can easily convert them into a webform.
For more information, please refer to ETSI TR 103 838 which requires a reporting webform.
Standard development organisations ("SDOs") should develop requirements for VDP accessibility and usability. This is important as they are at the basis of several regulations (and probably of the future harmonised European Standard ("hEN") for the Cyber Resilience Act).
Example of requirement to include in IoT cyber security standards:
The vulnerability disclosure policy and the "point of contact" must be accessible easily and publicly.
The VDP is almost always a public webpage. In several occasions, we found accessibility issues that could limit reporting and lead to a non-compliance.
We recommend all manufacturers to follow best practices for web accessibility. This includes user experience and making their VDP accessible on all platforms / browsers.
We would like to acknowledge David Rogers, the CopperHorse Limited team and the IoTSF for their yearly report on IoT VDP.
To continue this study, it would be interesting to look at statistics and trends regarding VDP accessibility and usability, by sector, by geography, by revenue, etc.
Another extension would be to analyse the content of the VDP and its compliance with standards and regulations.
Similar work could take place for product security advisories and their accessibility.
Additionally, the results of this study could contribute to the work on dynamic cyber security labels. These labels could check the VDP accessibility and usability as part of their requirements.
VDP is an important tool for improving the cyber security of consumer IoT products. In this study, we discovered that VDP accessibility is often overlooked. Indeed, having a VDP is not sufficient: it must be easy to access it and to submit reports.
We have identified multiple issues that could limit
the effectiveness of a VDP. We proposed several recommendations
including a global VDP strategy for all products, brands and markets
("one VDP"), a direct link in the footer menu and the implementation of
security.txt. Additionally, we believe that standard development
organisations should integrate the concepts of accessibility and
usability in their relevant standards.
At cetome, we promote VDP accessibility and usability in consumer IoT. This is an important area of work and it is now a regulatory requirement in several markets. For that purpose, we encourage all manufacturers to review their current approach and follow our recommendations.
To evaluate the accessibility of a vulnerability disclosure policy, we have developed a free "VDP Accessibility Scoring" tool accessible at cetome.com/vdp/score.
About this study
The study took place between July 2023 and January 2024. It was entirely self-funded.
cetome is an independent cyber advisory with a recognised expertise in IoT security. We work with IoT manufacturers to embed security-by-design in their products, train their teams and improve their cyber resilience. This includes the development of accessible and usable vulnerability disclosure policies.
About the Author
Dr. Cédric LÉVY-BENCHETON is the CEO and founder of cetome. Cédric has expertise in IoT security. He previously worked at ENISA, the European Union Cyber Security Agency.