Test Criteria

The tests of the products for their publication in the website OrientaTech are carried out from two points of view:

  • Technical verification, carried out by personnel with technical knowledge both Hardware and Software and specialists in its application in the social field.
  • Social validation of the products, carried out by volunteers and end users of the product with real needs of use.

Technical Verification

When establishing the technical verification procedure, it should be noted that there is no standard procedure for the valuation of social technology products because these technologies cover various functionalities and applications, however, a defined test methodology has been developed depending on the type of technology or product.

This methodology is based on the verification of some general criteria supported by indicators that allow to register a level of compliance of the criteria by the devices or applications.

 

Verification Criteria

The main criteria for technical verification are as follows:

  • Reliability: Probability that a system or device fulfills a certain function under certain conditions during a determined time.
  • Security: It is a combination of other criteria such as Authenticity, Confidentiality, Integrity and Availability that ultimately guarantee the non-vulnerability of a system.
  • Usability: It is the ability of the product to be understood, learned, and used easily and attractively.
  • Functionality: It is the ability of the product to meet and provide functions to meet explicit and implicit requirements when used under specific conditions.
  • Robustness: It refers to the quality of the materials and their ability to last through time.
  • Maturity: Time that the product takes in the market in a stable and sustainable way.
  • Performance: Time and responsiveness of the technological solution.
  • Scalability: A system's ability to change its dimension or configuration to adapt to changing circumstances.
  • Sustainability: Possibility of maintaining long term running without dependencies of third parties.
  • Portability: Having the ability to be executed on different platforms, with different hardware devices and operating systems.
  • Comprehensibility: The design of the product and application is based on elements or words, images and forms that serve to understand its operation.
  • Ease of acquisition: confirm whether the product is available or not in the market.
  • Ease of installation and configuration: Determines the degree of difficulty in the initial installation and configuration process.
  • Accessibility: It assesses the degree to which the product is accessible to people with basic access difficulties.
  • Consumption: determines the electrical consumption and battery life in the case of portable devices.
  • Weight and dimensions: determines the main physical characteristics of the product.
     

Technical staff

The team in charge of managing this technological verification, is made up of engineers specialized in the social application of technology from TECSOS Foundation.

 

Test Methodology

The tests are carried out in a uniform, consistent objective and in accordance with established regulations.

They are evaluated through predefined checklists based on the device being reviewed.

The evaluation criteria are evaluated by means of a scale, where each parameter is valued between 1 and 5 points and generates a final average score.

A technical verification report is made with the conclusions obtained according to the criteria assessed, the problems encountered and the recommendations proposed, highlighting points of improvement and highlights that the solution provides technically.

 

Test Environment

The tests are performed in a laboratory space designed for this purpose equipped with calibrated devices that allow different measures to be taken to estimate the parameters previously mentioned in the section of verification criteria.

In this way it is intended to carry out a validation of the products in their application in a real environment and to evaluate the sensations that the users have during their use.

 

Social validation

in parallel to the technical verification carried out by technology personnel, the product will be provided through CRE's and other associations to end users with similar needs to the actual users and potential customers of the product.

In this way it is intended to carry out a validation of the products in their application in a real environment and to evaluate the sensations that the users have during their use.

 

Validation criteria

The main social validation criteria are as follows:

  • Usability: It is the degree of effectiveness, efficiency and satisfaction with which specific users can achieve specific objectives, in specific contexts of use. It depends on the end user profiles of the product, device, service, or application.
  • Utility: It is the feature that verifies that the device actually covers the functionality it is intended to cover.
  • Understanding: the people who validate it must understand the processes that "occur" during the use of the product, in the mean that they require a response from the user.
  • Sensation of Safety / Reliability: This feature determines the perception of safety offered by the tool. In other words, regardless of whether the technology complies with security standards or other standards, it describes the person's feelings about the product, if they consider that their data may be vulnerable, if someone "sees" what they are doing, etc ...
  • Design / ergonomics: Opinion regarding the design distance from the part of usability, that is, only the reference to "like", "is friendly", adapts physically to the user, etc ...
  • Impact: It is a summary of the effect of the set of previous criteria on the end user, ultimately reflects to what extent the user's life changes thanks to the use of this product.
     

Social assessment staff

The validation of the products and services that will be tested and analyzed from the TECSOS Foundation seeks a complete view of the operation.

In addition to the technical staff, team of volunteers and social organizations, are involved in tests of greater scope, which evaluates the impact on real users.

Therefore, it is only necessary to put yourself in the place of "customers" of the product, and guided by checkpoints and evaluation questionnaires, the degree of efficiency, efficiency and satisfaction with which the designed devices can be successfully used .

The figures of "testers" are, on the one hand, volunteers of the Spanish Red Cross and, on the other, groups of elderly people, with disabilities or other groups of interest.

 

Test Methodology

The tests will be carried out in the form of Validation Sessions with users within the laboratory in some cases and tests of greater long duration in real environments in other cases.

The scoring system for the social validation criteria is analogous to that of social verification in such a way that the different verification criteria are evaluated by means of a scale, where each parameter is valued between 1 and 5 points and a definitive average score is generated.

A social validation report is made with the conclusions obtained according to the assessed criteria, the problems encountered and the recommendations that are proposed, highlighting points of improvement and highlights that the solution contributes to the users.