Project Design and Management — Alessandro Carella

This project focused on improving the user experience for Fondo Ambiente Italiano (FAI) , a non-profit organization dedicated to protecting and promoting Italy's natural and cultural heritage. The primary objective was to assess the desirability of potential solutions that address user needs, particularly for active seniors navigating the FAI website to discover and participate in cultural events.

Through comprehensive research and analysis, me and my team identified user groups, validated their needs, and developed three innovative solutions: an interactive map for event discovery, a customizable email notification system, and an AI-powered conversational chatbot. Each solution was tested and assessed using quantitative metrics including time efficiency, user satisfaction, engagement levels, and technical feasibility.

This project provided insights into the complete product design and management lifecycle, from user research to solution implementation and assessment. One of the most significant learnings was the importance of combining top-down and bottom-up research approaches to gain comprehensive user insights. The website analysis gave us FAI's organizational perspective, while social network analysis and user comments revealed genuine pain points that users experienced during major events like FAI Autumn Days.

I learned the critical value of structured ideation methods. Techniques like Round-Robin Brainstorming , Brain-Writing , and the Nominal Group Technique helped our team generate diverse ideas while mitigating common brainstorming pitfalls such as groupthink and dominant personalities. The integration of AI tools like ChatGPT for idea expansion demonstrated how generative AI can augment human creativity when used strategically with well-engineered prompts.

The need hypothesis validation process taught me the importance of designing testable, precise, and measurable experiments. We structured our hypotheses using the format "We believe that... So if... Then..." which forced us to think critically about validation criteria before conducting experiments. This approach helped us avoid the common pitfall of confirmation bias and ensured we collected meaningful data.

User testing revealed surprising insights that challenged our initial assumptions. For instance, we expected the UK National Trust website to outperform FAI's site due to its additional features, but users actually found it more confusing. This taught me that more features do not always equal better user experience, a critical lesson in product design. Users simply wanted to find relevant events quickly, and excessive tools sometimes hindered rather than helped.

The assessment methodology was particularly enlightening. Creating a Quality Index that combined time metrics with user satisfaction (using logarithmic transformation) provided a nuanced view of solution effectiveness. The final scoring formula that incorporated time, satisfaction, engagement, and feasibility demonstrated how multi-dimensional assessment can lead to more informed decision-making than relying on a single metric.

Working with real users through the Thinking Aloud technique revealed the gap between what users say they want and what they actually need. Their spontaneous comments during navigation tasks provided authentic insights that traditional surveys might have missed. This experience reinforced the value of observational research methods in UX design.

I also learned about the challenges of implementing AI solutions in real-world contexts. While our VirgilioGPT chatbot scored well on engagement and satisfaction, its feasibility was limited by slow response times when searching the internet. This highlighted the importance of balancing innovation with practical constraints, and the need to consider technical limitations early in the design process.

Finally, this project demonstrated the importance of consensus-building and majority-voting systems in team decision-making. We successfully navigated complex choices by combining structured discussion with democratic voting, ensuring all team members felt heard while maintaining project momentum.

The research employed a mixed-method approach combining top-down and bottom-up strategies to identify users and their needs. The team evaluated seven potential research methods using a structured assessment matrix that scored each method on feasibility, efficiency, and effectiveness. This analysis led to the selection of three primary methods: website analysis, social network analysis, and benchmarking.

The website analysis revealed critical insights into FAI's existing user base through examination of membership categories. The Italian version of the website proved significantly richer in content than the English version, featuring prominent calls-to-action for membership enrollment. Analysis of FAI's financial statements showed that membership represents the most significant income source after donations, validating the importance of understanding and serving these user segments effectively.

The team conducted a detailed analysis of FAI's Facebook page, examining posts from October to identify engagement patterns and demographic information. Through systematic cataloging of post themes and quantitative analysis of reactions and comments, the research revealed that users aged 30-40, 50-60, and 60-70 showed the highest engagement levels. Critical insights emerged from analyzing negative comments during FAI Autumn Days, which revealed specific pain points including unclear ticket pricing, poor coordination, and long wait times without reservation systems.

Comparative analysis with organizations like the UK's National Trust and France's Centre des Monuments Nationaux provided valuable insights into best practices for heritage site management. The UK's National Trust, with over 5.6 million members and annual income exceeding £680 million, demonstrated successful engagement strategies. This analysis identified potential user groups not yet fully served by FAI, including eco-tourists, photographers and artists, and digital content consumers, while also revealing opportunities such as implementing restaurants at FAI locations.

The team identified six primary user categories: Professionals with Cultural and Travel Interests, Active Seniors, Cultural Enthusiasts, Foreign Tourists, Experience-Seeking Tourists, and Recreation and Fitness Groups. Each group had distinct needs ranging from personalized trip customization to enhanced event comparison tools.

Using Brain-Writing technique combined with the Nominal Group Technique, the team formulated testable need hypotheses following the structure "We believe that... So if... Then...". Each hypothesis was designed to be testable, precise, and measurable. For example, one key hypothesis stated: "We believe that active seniors with an interest in culture and travel members of FAI are not checking the new events because of the high number of proposals. So if we select only the ones that might be attractive to them based on interests and location and show the smaller list to them, then more than half would read it and ask more questions about it."

Through systematic experimentation, three needs were validated while two were not. The validated needs included better event comparison tools based on location and interests, more organized trip possibilities, and the use of FAI locations for celebrations like weddings. However, the local market concept and enhanced experiences for fitness enthusiasts did not validate, with less than 20% of test participants showing genuine interest in these features. This validation process helped focus resources on solutions that would genuinely serve user needs.

The team employed multiple divergent thinking methods to generate solution ideas. Free-Wheeling Brainstorming produced initial concepts, while the Thinking Aloud technique during user testing sessions revealed users' spontaneous suggestions. Benchmarking with similar organizations identified proven features, and ChatGPT was used with carefully engineered prompts to expand the solution space. The team used the Prompt Canvas framework to structure effective prompts, ensuring comprehensive exploration of AI-driven solutions.

The solution development process converged on three main approaches. The interactive map solution provides a dynamic, user-friendly interface showcasing FAI locations with filtering capabilities based on user preferences. This solution was inspired by successful implementations on benchmarked sites like New Zealand's National Trust, where the map serves as a primary navigation tool on the homepage. The customizable email solution creates a direct communication channel where users submit natural language queries about their interests, geographical preferences, and travel willingness, receiving personalized event recommendations via email. The conversational chatbot solution, named VirgilioGPT, leverages GPT-4 to provide human-like interactions, helping users discover events through natural conversation rather than traditional website navigation.

The team developed sophisticated metrics for solution assessment. The Quality Index formula combined time efficiency with user satisfaction using logarithmic transformation to avoid sparse scoring. The final assessment formula incorporated four normalized dimensions: time (inverted, as lower is better), satisfaction, engagement, and technical feasibility. This multi-dimensional approach provided a comprehensive evaluation framework that captured both quantitative performance and qualitative user experience factors.

Solutions were tested using A/B testing methodology with heterogeneous user groups representing different age ranges and backgrounds. For the map and baseline tests, users were asked to find specific events on different websites while time and satisfaction metrics were collected. The email solution was tested by having users navigate FAI's event page, then write descriptions of interesting events, which were processed using ChatGPT to generate personalized recommendations. The chatbot testing employed a split-screen framework with the FAI website on one side and VirgilioGPT chat on the other, allowing users to interact naturally with the AI assistant while browsing.

The assessment revealed clear performance differences among the three solutions. The interactive map achieved the highest overall score of 13.16, demonstrating superior time efficiency with an average of 0.71 units, the highest satisfaction rating of 4.38 out of 5, and strong engagement at 4.50. Technical feasibility was rated at 5.00, indicating the solution could be implemented with existing technology. The email solution scored 6.78 overall, with moderate satisfaction (3.69) and good feasibility (4.00), but longer average task completion time of 4.42 units. The chatbot solution scored 7.56, achieving the highest engagement rating of 5.00 and strong satisfaction of 4.45, though feasibility was lower at 3.00 due to technical constraints around response time.

User feedback revealed several important patterns. Many users commented that events seemed too targeted toward families, suggesting an opportunity to diversify event marketing. The website's event page layout, with numerous events displayed simultaneously using large images and long titles, created disorientation rather than clarity. Several users expressed desire for event categorization by type, such as museum visits versus forest walks, and requested community-voted "best event" features. Users also sought more information about accessibility for elderly and disabled visitors, including details about host structures, safety ratings, and proximity to healthcare facilities.

The map solution surprised the team by outperforming the UK National Trust website, which had been assumed to be superior due to additional features. Users found that excessive tools actually created confusion, as they simply wanted to find relevant events quickly. The email solution showed promise for collecting user contact information and enabling future engagement through periodic newsletters, though requiring users to leave the website was identified as a limitation. The chatbot solution generated strong user enthusiasm, with some users preferring conversation over traditional website navigation. However, VirgilioGPT's need to search the internet made it too slow for practical deployment without further optimization. The team noted that training a GPT specifically on FAI website content could resolve this issue, and identified commercial solutions that offer this capability.

The project faced several significant challenges that provided valuable learning opportunities. Recruiting appropriate test participants proved more difficult than anticipated, particularly for targeting the active senior demographic. The team adapted by conducting outreach through community centers and leveraging personal networks. Technical challenges emerged during chatbot development, as VirgilioGPT's web search functionality created latency. This required problem-solving and led to the discovery of commercial solutions that could address the issue. Balancing innovation with practical constraints required careful consideration throughout the solution development process.

Our five-person team successfully navigated complex decision-making through structured processes including Round-Robin Brainstorming, Brain-Writing, and Nominal Group Technique. These methods ensured all voices were heard while maintaining project momentum. The team effectively combined individual expertise in data science, UX research, and technical implementation. Regular meetings and clear role assignments helped maintain coordination across research, development, and assessment activities. The integration of AI tools like ChatGPT demonstrated how teams can augment their capabilities while maintaining human judgment in critical decisions.

The structured approach to user research, need validation, and solution assessment proved highly effective. The decision to evaluate and select research methods using explicit criteria prevented common pitfalls and ensured efficient resource split. The need hypothesis framework forced thinking about validation criteria before conducting experiments, improving the quality of insights. The multi-dimensional assessment formula provided understanding beyond simple rankings, revealing trade-offs between different solution characteristics. This methodology could be effectively applied to other UX design projects and product development initiatives.