Task Description: You are a Data Engineer tasked with designing and implementing a proof- of concept Big Data analytics solution for a city’s transport authority. Scenario: The city council wants to analyse urban mobility patterns using

DEGREE: BSc Computer Science and Digitisation

Module: Big Data Analytics using AI

Assignment Title: SmartCity Urban Mobility Analysis using Hadoop and

Predictive AI

Assignment Type: Report

Word Limit: 3000 words (+/- 300)

Weighting: 100%

Issue Date: 4/9/2025

Submission Date: 30/9/2025

Feedback Date: 21/10/2025

Plagiarism:

When submitting work for assessment, students should be aware of the

InterActive/Canvas guidance and regulations in concerning plagiarism. All submissions should be your own, original work. Please note that you must not submit the same assignment for two different modules within your course.

You must submit an electronic copy of your work. Your submission will be electronically checked.

Harvard Referencing:

The Harvard Referencing System must be used. The Wikipedia, UKEssays.com or similar websites must not be used or referenced in your work.

Introduction

 

The goal of this assignment is to provide you with hands-on experience in designing and implementing a Big Data analytics solution that incorporates a predictive AI component. You will address a hypothetical smart city challenge by using the Hadoop ecosystem to process large-scale data and derive actionable insights for urban planning. This assignment requires you to design a solution using Hadoop, HDFS, YARN, and MapReduce to analyse transportation data. The final step involves using the processed data to train a simple predictive model, thereby connecting Big Data processing with AI applications. This will help you understand the end-to-end pipeline from raw data to business intelligence in a modern context.

Scenario: The city council wants to analyse urban mobility patterns using data from road sensors, taxi trips, or public transit records. The objective is to identify congestion hotspots, understand their causes, and predict future traffic patterns to enable proactive traffic management and better infrastructure planning.

Phase 1: Conceptual Design & Architecture (LO 1, LO 2) – 20% of Total Grade

1.  Business Context and Problem Statement (5%)

•        Describe the smart city scenario, focusing on the challenges of urban mobility.

•        Define a clear problem statement. For example: “To analyse historical traffic data to predict the likelihood of traffic congestion at key intersections based on time of day and day of the week”.

•        Explain how solving this problem provides tangible value to the city (e.g., reduced commute times, lower pollution, improved public safety).

2.  Hadoop Ecosystem and Architecture (15%)

•        Explain why a Big Data approach is necessary for this scenario.

•        Identify the roles of HDFS, YARN, and MapReduce in your proposed solution.

•        Justify your choice of these components for the defined problem.

•        Create a clear architectural diagram illustrating how data flows from source to HDFS, is processed by MapReduce managed by YARN, and is then used for analysis.

Phase 2: Implementation & Analysis (LO 3) – 50% of Total Grade

 

1.  Data Acquisition & Preparation (5%)

•        Select a suitable public dataset representing urban mobility (e.g., taxi trip records, traffic sensor data). Platforms like Kaggle or city-specific open data portals are good sources.

•        Describe the dataset’s structure, size, and key attributes relevant to your problem statement.

2.  Hadoop Environment and Data Ingestion (10%)

•        Set up a local single-node Hadoop cluster (e.g., using the official Apache Hadoop binaries or a Docker image).

•        Document the key steps of your setup process.

•        Load your chosen dataset into HDFS. Provide the commands and screenshots showing the data successfully stored in HDFS.

3.  Data Processing with MapReduce (20%)

•        Write a MapReduce program in Java or Python to process the data. Your programmust perform data cleaning and feature engineering to prepare it for the predictive model.

Example tasks: calculate average trip duration per route, count vehicle flow per_• hour, or identify other relevant features from the raw data.

•Explain the logic of your Mapper and Reducer classes and include the well- commented source code in your report’s appendix.

4.  Predictive Analysis and Visualization (15%)

•        Export the processed data from HDFS.

•        Use the processed data to train a simple predictive model. You can use a library like Scikit-learn in Python to build a classification or regression model that addresses your problem statement.

•        Analyze and interpret the output of your MapReduce job and your predictive model.

•        Create meaningful visualizations (e.g., graphs showing congestion by time of day, a confusion matrix for your model) to present your findings.

Phase 3: Reflection and Documentation (LO 1, LO 2, LO 3) – 30% of Total Grade

1.  Critical Reflection (10%)

•        Reflect on the key challenges you encountered during implementation (e.g., data cleaning, debugging MapReduce, model accuracy) and how you addressed them.

•        Critically discuss the performance and scalability of your MapReduce solution.

Could it be optimized (e.g., by using a Combiner)?

2.  Final Report Documentation (20%)

•        Compile a detailed, professional report of no more than 3000 words documenting the entire project.

•        The report must be well-structured with clear headings, proper grammar, and academic language.

•        

Ensure all phases (Conceptual Design, Implementation, Reflection) are thoroughly covered, including diagrams, code snippets, commands, and visualizations to support your work.

•        Include a bibliography using the Harvard referencing style.

 

Submission Guidelines:

•        Document Format: Submit your assignment as a single document following theBSBI assignment template provided in Canvas.

•        Writing Quality: Ensure clear and concise writing with proper grammar and spelling. Use headings and subheadings to organize your work logically according to the tasks outlined above.

•        Visuals: Include visuals like diagrams (process flow, conceptual model sketches), tables (data assumptions, results), and graphs (simulation output) where appropriate to enhance understanding.

•        Task Coverage: Address each part thoroughly, demonstrating your understanding of Big Data concepts and their application to the business scenario.

•        Implementation Details: Provide relevant examples and details of your model implementation, including code snippets, commands, and calculations.

•        Referencing Style: Use Harvard referencing style for your bibliography.

•        Discussion: Discuss your findings, insights, and the implications of your recommendations. Reflect on the challenges faced and how you overcame them.

•        Submission: Submit your assignment electronically (Canvas) by the specified deadline.

GUIDANCE ON ASSESSMENT

All materials must be properly referenced under Harvard conventions. The length required is 3000 words with tasks equally weighted. The writing style should be formal academic / report writing style with in-text referencing to support your comments and observations. Originality, quality of argument and good structure are required. The report should demonstrate sound understanding and ability to apply knowledge and theory of Simulation Techniques. Additional marks being awarded for juxtaposition and insight of issues.

Grading Criteria

Generic Criteria       90 – 100

Knowledge of contexts, concepts, technologies and     Exceptional breadth and

processes          depth of knowledge of

The extent to which:               contextual and theoretical

issues, some of which are relevant contextual or theoretical at the forefront of the issues are identified, defined and discipline, and their described

relationship to a range of

historical or contemporary   historical and practices are identified, defined                   contemporary practices and described

appropriate technologies,

methods and processes are Exceptional knowledge of identified defined and described           a range of relevant specialist techniques and processes

Understanding through         Exceptional application of application of knowledge    a range of research The degree to which research                        methodologies to projects methods are demonstrated:                  and problems and hypotheses, with evidence relevant knowledge and              of highly focused information is compared,     independent thought and contrasted, manipulated,     some new insights into translated and interpreted                       the subject knowledge and information is

selected, analysed, synthesized and evaluated in order to      Exceptional ability to

generate creative ideas, practices,             produce a range of

solutions, arguments or        creative practices and to hypotheses     critically evaluate them in

a wider context, generating sustainable arguments and highly effective and individual results

80 – 89

Outstanding breadth and depth of knowledge of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Extensive knowledge of a range of relevant specialist techniques and processes

Systematic and thorough application of a range of research methodologies

to projects and problems and hypotheses, with evidence of highly focused independent thought and some new insights into the subject

Outstanding ability to produce a range of creative practices and to critically evaluate them in a wider context , generating sustainable arguments and highly effective and original results

70 – 79

A breadth and depth of knowledge of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Significant knowledge of a range of relevant specialist techniques and processes

Rigorous application of a range of research methodologies to projects , problems and hypotheses with evidence of highly focused independent thought and

critical analysis

Strong ability to produce a range of creative practices and to critically evaluate them in a wider context, generating sustainable arguments and highly

effective results

60 – 69

Confident knowledge of a range of contextual and theoretical issues, some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Confident knowledge of a range of relevant specialist techniques and processes

Confident ability to apply a range of research methodologies to projects, problems and hypotheses with clear evidence of independent thought and critical analysis

Strong ability to produce a range of creative practices and to evaluate them in a wider context , generating effective results

50 – 59

Familiar with a range of contextual and theoretical issues, at least some of which are at the forefront of the discipline, and their relationship to a range of historical and contemporary practices

Sound knowledge of a range of relevant specialist techniques and processes

Sound ability to apply a range                  of                    research methodologies to projects, problems and hypotheses and                 to demonstrate independent                         thought

and critical analysis

Sound ability to produce arange of creative practices and to evaluate them in a wider context, generating effective results

40 – 49

Familiar with a range of contextual and theoretical issues and their relationship to a

range of historical and

contemporary practices

Adequate knowledge of a range of relevant specialist techniques and processes

Competent ability to apply a range of research methodologies to projects, problems and hypotheses with some element of independent thought and critical analysis

Competent ability to produce a range of creative practices and evaluate them in a wider context to generate effective results

30 – 39

Some knowledge of a range of contextual and theoretical issues and their relationship to arange of historical and contemporary practices

Limited knowledge of a range ofrelevant specialist techniques and processes

Ability to apply a limited range of research methodologies to projects, problems and hypotheses with little evidence of independent thought or critical analysis

Limited ability to produce a range of creative practices and to evaluate them in a wider context to generate effective results

0-29

Limited knowledge of contextual and theoretical issues and their relationship to a range of historical and contemporary practices

No                   significant knowledge of a range of relevant specialist techniques    or processes No significant ability to apply research methodologies to projects, problems and hypotheses, and no evidence of

independent

thought or critical analysis

No significant ability to produce a range of creative practices or to evaluate them in a wider context to generate effective results

5

Application of technical and professional skills The degree to which:

appropriate materials and media are selected, tested and utilised to realise and present ideas and solutions appropriate technologies, methods and processes are demonstrated

transferable, professional skills are effectively demonstrated self management and independent learning are demonstrated Exceptional,individual andfluentapplication ofa range of specialist

practical and technical skills

Outstanding accomplishment of a range of advanced transferable and professional skills applied to complex situations and problems

Exceptional ability to manage own learning in a sustained manner and to critically evaluate own progress, making use of a wide range of feedback sources Accomplished,original andfluentapplication ofa range of specialist

practical and technical skills

Outstanding accomplishment of a range of advanced transferable and professional skills applied to complex situations and problems

Outstanding ability to manage own learning in a sustained manner and to critically evaluate own progress, making use of a wide range of feedback sources

Accomplished and original applicationof a range of specialist practical and technical skills

Accomplished application of advanced transferable and professional skills to complex situations and problems

Very high ability to manage own learning in a sustained manner and critically evaluate own progress making effective use of feedback

 

Confident and imaginative application of a range of specialist practical and technical skills

Confident application of advanced transferable and professional skills to challenging situations and problems

Strong ability to manage own learning in a sustained manner and to critically evaluate own progress making effective use of feedback

Sound application ofa range of specialist practical and technical skills

Sound application of advanced transferable and professional skills

Sound ability to manage own learning in a sustained manner and critically evaluate own progress making effective use of feedback Competent application ofa range of specialist practical and technical skills

Competent application of advanced transferable professional skills

Competent ability to manage own learning in a sustained manner and make effective use of feedback

Basic application of a range of specialist practical and technical skills

Limited application of advanced transferable and professional skills

Basic ability to manage own learning in a sustained manner and make use of feedback Rudimentary application of a range of specialist practical and technical skills

Ineffective application of advanced transferable and professional skills

Evidence of a basic ability to manage own learning

6