Senior Data Engineer, Data Engineering

Walt Disney Co.
Walt Disney Co.

Job Overview

Job Summary:We have created a new Big Data Platforms group within Disney Direct-To-Consumer and International (DTCI) technology with the skills, drive, and passion to innovate, create, and succeed in enabling a Direct to Consumer Strategy for ESPN, Disney and ABC products. We are here to disrupt and start a cultural revolution that can lead a revolution in the application of data and analytics across The Walt Disney Company, focused on building self-serve analytics, advanced analytics using machine learning methods Deep User Understanding, Audience Segmentation for Linear to Digital Ad Sales. We need an experienced Data Engineer who drive multiple data initiatives applying innovative architecture that can scale in the cloud. We are looking for a creative and talented individual who loves to design a scalable platform which scale at peta-byte level and extract value from both structured and unstructured real-time data. Specifically, we are looking for a technology leader to build a highly scalable and extensible Big Data platform which enables collection, storage, modeling, and analysis of massive data sets from numerous channels. You must be self-driven to continuously evaluate new technologies, innovate and deliver solutions for business-critical applications with little to no oversight from management team.The internet-scale platforms that you design and build will be a core assets in the delivering the highest quality content to over 150MM+ consumers on monthly basis. This is an opportunity to fundamentally evolve how DTCI delivers content and monetizes our audiences.Responsibilities:* Build cool things – Build scalable analytics solution, including data processing, storage, and serving large-scale data through batch and stream, analytics for both behavioral & ad revenue through digital & non-digital channels.* Harness curiosity – Change the way how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.* Innovate and inspire – Think of new ways to help make our data platform more scalable, resilient and reliable and then work across our team to put your ideas into action.* Think at scale – Lead the transformation of a peta-byte scale batch based processing platform to a near real-time streaming platform using technologies such as Apache Kafka, Cassandra, Spark and other open source frameworks.* Have pride – Ensure performance isn’t our weakness by implementing and refining robust data processing using Python, Java, Scala and other database technologies such as RedShift or Snowflake .* Grow with us – Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways which make other groups jealous.* Lead and coach – Mentor other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.* ML First – Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.* Build and Support – Embrace the DevOps mentality to build, deploy and support applications in cloud with minimal help from other teamsBasic Qualifications:* Not your first rodeo – Have 4+ years of experience developing data driven application using mix of languages (Java, Scala, Python, SQL etc.) and open source frameworks to implement data ingest, processing, and analytics technologies.* Data and API ninja -You are also very handy with big data framework such as Hadoop, Apache Spark, No-SQL systems such as Cassandra or DynamoDB, Streaming technologies such as Apache Kafka; Understand reactive programming and dependency injection such as Spring to develop REST services.* Have a technology toolbox – Hands on experience with newer technologies relevant to the data space such as Spark, Airflow, Apache Druid, Snowflake (or any other OLAP databases).* Cloud First – Plenty of experience with developing and deploying in a cloud native environment preferably AWS cloud.* Embrace ML – Work with data scientists to operationalize machine learning models and build apps to make use of power of machine learning.* Problem solver – Enjoy new and meaningful technology or business challenges which require you to think and respond quickly.* Passion and creativity – Are passionate about data, technology, & creative innovationPreferred Qualifications:* Prior experience building internet scale platforms – handling Peta- byte scale data, operationalizing clusters with hundreds of compute nodes in cloud environment.* Experience in operationalizing Machine Learning workflows to scale will be a huge plus as well.* Experience with Content Personalization/Recommendation, Audience Segmentation for Linear to Digital Ad Sales, and/or Analytics* Experience with open source such as Spring, Hadoop, Spark, Kafka, Druid, Kubernetes.* Experience in working with Data Scientists to operationalize machine learning models.* Proficiency with agile development methodologies shipping features every two weeks. It would be awesome if you have a robust portfolio on Github and / or open source contributions you are proud to shareRequired Education* Bachelor’s degree or better in Computer Science or a related technical field or equivalent job experiencePreferred Education* Masters in Computer Science or similar is preferredResponsibilities:* Build cool things – Build scalable analytics solution, including data processing, storage, and serving large-scale data through batch and stream, analytics for both behavioral & ad revenue through digital & non-digital channels.* Harness curiosity – Change the way how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.* Innovate and inspire – Think of new ways to help make our data platform more scalable, resilient and reliable and then work across our team to put your ideas into action.* Think at scale – Lead the transformation of a peta-byte scale batch based processing platform to a near real-time streaming platform using technologies such as Apache Kafka, Cassandra, Spark and other open source frameworks.* Have pride – Ensure performance isn’t our weakness by implementing and refining robust data processing using Python, Java, Scala and other database technologies such as RedShift or Snowflake .* Grow with us – Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways which make other groups jealous.* Lead and coach – Mentor other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.* ML First – Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.* Build and Support – Embrace the DevOps mentality to build, deploy and support applications in cloud with minimal help from other teamsBasic Qualifications:* Not your first rodeo – Have 4+ years of experience developing data driven application using mix of languages (Java, Scala, Python, SQL etc.) and open source frameworks to implement data ingest, processing, and analytics technologies.* Data and API ninja -You are also very handy with big data framework such as Hadoop, Apache Spark, No-SQL systems such as Cassandra or DynamoDB, Streaming technologies such as Apache Kafka; Understand reactive programming and dependency injection such as Spring to develop REST services.* Have a technology toolbox – Hands on experience with newer technologies relevant to the data space such as Spark, Airflow, Apache Druid, Snowflake (or any other OLAP databases).* Cloud First – Plenty of experience with developing and deploying in a cloud native environment preferably AWS cloud.* Embrace ML – Work with data scientists to operationalize machine learning models and build apps to make use of power of machine learning.* Problem solver – Enjoy new and meaningful technology or business challenges which require you to think and respond quickly.* Passion and creativity – Are passionate about data, technology, & creative innovationPreferred Qualifications:* Prior experience building internet scale platforms – handling Peta- byte scale data, operationalizing clusters with hundreds of compute nodes in cloud environment.* Experience in operationalizing Machine Learning workflows to scale will be a huge plus as well.* Experience with Content Personalization/Recommendation, Audience Segmentation for Linear to Digital Ad Sales, and/or Analytics* Experience with open source such as Spring, Hadoop, Spark, Kafka, Druid, Kubernetes.* Experience in working with Data Scientists to operationalize machine learning models.* Proficiency with agile development methodologies shipping features every two weeks. It would be awesome if you have a robust portfolio on Github and / or open source contributions you are proud to shareRequired Education* Bachelor’s degree or better in Computer Science or a related technical field or equivalent job experiencePreferred Education* Masters in Computer Science or similar is preferredAbout Direct-to-Consumer and International:About The Walt Disney Company:The Walt Disney Company, together with its subsidiaries and affiliates, is a leading diversified international family entertainment and media enterprise with the following business segments: media networks, parks and resorts, studio entertainment, consumer products and interactive media. From humble beginnings as a cartoon studio in the 1920s to its preeminent name in the entertainment industry today, Disney proudly continues its legacy of creating world-class stories and experiences for every member of the family. Disney’s stories, characters and experiences reach consumers and guests from every corner of the globe. With operations in more than 40 countries, our employees and cast members work together to create entertainment experiences that are both universally and locally cherished.Apply Now Apply LaterTHE WALT DISNEY COMPANY PRIVACY NOTICE FOR VENDORS, CONTRACTORS, SERVICE PROVIDERS AND OTHER EXTERNAL RESOURCESLast Updated: June 4, 2020The Walt Disney Family of Companies (“we” or “us”) values your trust and is committed to the responsible management, use and protection of Personal Information. Personal Information is any information that identifies (whether directly or indirectly) a particular individual (defined more fully below). This privacy notice describes our treatment of information related to the collection, use and storage of information of Vendors, Employment and Onboarding, Contractors (External), Training or Other outside Resources (“VECTOR Privacy Notice”) via publicly available websites and applications (including those with registration). Please read this VECTOR Privacy Notice carefully prior to submitting any Personal Information.This VECTOR Privacy Notice applies wherever it is posted. This VECTOR Privacy Notice does not apply to our processing of consumer or guest information provided or collected on sites where our Global Privacy Policy is posted. This VECTOR Privacy Notice also does not apply to employee data processed in accordance with an applicable employee privacy notice.Table of Contents:* Who we are* Information We Collect* Use of Information* Disclosure of Personal Information* Data Retention* Access and Correction* Security* Links to Third-Party Websites* Data Transfers and Applicable Law* Additional Information for the EEA Only* Changes to the VECTOR Privacy Notice* Contact UsWho we areA member of The Walt Disney Family of Companies, which includes many different brands, will be the Data Controller for your information under this VECTOR Privacy Notice. The list of relevant Data Controllers can be found below. The member of the The Walt Disney Family of Companies that is the Data Controller for your information is responsible for and may use your information for the purposes described in this VECTOR Privacy Notice. Other members of The Walt Disney Family of Companies may have access to your information where they perform services on behalf of the Data Controller(s) (as a Data Processor), and unless prohibited under applicable law for use on their own behalf (as a Data Controller) for the purposes described in this VECTOR Privacy Notice.This VECTOR Privacy Notice also applies to the collection and use of your Personal Information by any member of The Walt Disney Family of Companies that may need access to information you have provided.Information We CollectInformation You ProvideWe may collect Personal Information (as defined in this VECTOR Privacy Notice) and anonymous information about you which is relevant to our business relationship with you, such as:* Registration information you provide when you create an account, or link your profile on a third-party site or platform with your registration account, such as your first name and username, country of residence, gender, date of birth, email address, and password;* Information sent either one-to-one or within a limited group using our message, chat, post, or similar functionality, where we are permitted by applicable law to collect this information;* Information in connection with an employment application, including contact information, work authorization status, CV/resume, previous work experience and education information, skills, references, work-related licenses, permits and certifications; any other information you elect to provide to us (e.g., awards or professional memberships);* Location information, including location information provided by a mobile or other device interacting with one of our sites or applications (including through beacon technologies), or associated with your IP address, with your consent and where we are permitted by law to process this information;* Activity information about your use, and the use by any person(s) you authorize through your account, of our sites and applications, such as the content you view or post, how often you use our services, and your preferences; and* Usage, viewing, technical, and device data when you visit our sites, use our applications on third-party sites or platforms, or open emails we send, including your browser or device type, unique device identifier, and IP address.Cookies and Similar TechnologiesONLINE TRACKING TECHNOLOGIES AND ADVERTISINGWe and certain service providers operating on our behalf collect information about your activity, or activity on devices associated with you, on our sites and applications using tracking technologies such as cookies, Flash cookies, pixels, tags, software development kits, application program interfaces, and Web beacons. Definitions for some of the tracking technologies listed, as well as information about your choices with respect to them, are available below. This tracking data may be used for many purposes including, for example, to:* Provide useful features to simplify your experience when you return to our sites and applications (for example, remembering your language preference);* Deliver relevant content based on your preferences, usage patterns and location;* Monitor, evaluate, and optimize the use and operation of our sites and applications; and* Analyze traffic on our sites and on the sites of third parties.We may collect information whether or not you are logged in or registered, and may associate this tracking data with your registration account (if you have one), in which case we will treat it as Personal Information. Service providers that collect tracking data on our behalf may provide an opportunity for you to choose not to be tracked online.Advertisers and third parties also may collect information about your activity on our sites and applications, on devices associated with you, and on third-party sites and applications using tracking technologies. Tracking data collected by these advertisers and third parties is used to decide which ads you see both on our sites and applications and on third-party sites and applications.You may choose not to receive targeted advertising from many ad networks and partners, data exchanges, and marketing analytics and other digital advertising and marketing service providers. You may also be able to choose to control targeted advertising on other websites and platforms that you visit. In addition, you may also choose to control targeted advertising you receive within applications by using the settings and controls on your devices (for example, by re-setting your mobile device’s advertising identifier and/or opting out of interest based ads). We adhere to the Digital Advertising Alliance’s Self-Regulatory Principles for Online Behavioral Advertising.Examples of online tracking technologies include:* Cookies. Cookies are pieces of information that a website places on the hard drive of your computer when you visit the website. Cookies may involve the transmission of information from us to you and from you directly to us, to another party on our behalf, or to another party in accordance with its privacy policy. We may use cookies to bring together information we collect about you. You can choose to have your computer warn you each time a cookie is being placed on your device, or you can choose to block all cookies. You do this through your browser settings. Each browser is a little different, so look at your browser Help menu to learn the correct way to modify your cookies. If you turn cookies off, you won’t have access to many features that make your experience more efficient, and some of our services will not function properly.* Flash cookies. We may use local shared objects, sometimes known as Flash cookies, to store your preferences or display content based upon what you view on our site to personalize your visit. Our third-party service providers also may use Flash cookies to collect and store information. Flash cookies are different from browser cookies because of the amount of, type of, and manner in which data is stored. Cookie management tools provided by your browser will not remove Flash cookies. Learn how to manage privacy and storage settings for Flash cookies. If you disable Flash cookies, you won’t have access to many features that make your experience more efficient, and some of our services will not function properly.* Web beacons. Web beacons are small pieces of data that are embedded in images on the pages of sites. Web beacons may involve the transmission of information directly to us, to another party on our behalf, or to another party in accordance with its privacy policy. We may use web beacons to bring together information we collect about you.DO NOT TRACKWeb browsers can transmit Do Not Track signals that indicate that a user does not wish to have activity tracked. Currently, no universally accepted standard exists for how to interpret such signals, although work to create consensus is ongoing. Learn about Do Not Track.SENSITIVE INFORMATIONExcept as specifically requested or legally required (e.g., in the context of diversity hiring practices, health data collected pursuant to law or the demand of a government entity), we ask that you avoid submitting information which may qualify as sensitive or “special category” information under applicable law. Sensitive Personal Information may include Personal Information regarding your racial or ethnic origin; your political opinions, religious beliefs or other beliefs of a similar nature; whether you are a member of a trade union; data regarding your physical or mental health or condition; your sexual orientation; and data regarding the commission or alleged commission of any criminal offense.Use of InformationA noted above, the member of The Walt Disney Family of Companies that is the Data Controller for your information is responsible for and may use your information for the purposes described in this VECTOR Privacy Notice. Other members of The Walt Disney Family of Companies may have access to your information where they perform services on behalf of the Data Controller (as a Data Processor) and, unless prohibited under applicable law, for use on their own behalf (as a Data Controller) for the purposes described in this VECTOR Privacy Notice. Consistent with applicable law and choices and controls that may be available to you, we may use information collected from you, or from devices associated with you for our business purposes, including to:* Provide you with the services you request, view, or engage with;* Communicate with you about your account or transactions with us and send you information or request feedback about features on our sites and applications or changes to our policies;* Personalize content and experiences;* Provide you with targeted advertising based on your activity on our sites and applications and on third-party sites and applications;* To learn more about how we use your information for personalization and tracking, please see Online Tracking Technologies and Advertising above.* Manage our suppliers, independent contractors, service providers and others throughout the supply chain;* Organize and implement tasks in preparation of or to perform existing contracts;* Evaluate activities at our facilities, including compliance with applicable policies, as well as health and safety rules in place;* Grant you access to our training and collaboration modules allowing you access to certain materials and services you are entitled to receive;* Manage and support our IT resources, including infrastructure management and business continuity;* Manage diversity information and career progression tracking;* Preserve our economic interests and ensure compliance and reporting (such as complying with our policies and local legal requirements, tax and deductions, conducting audits, responding to complaints, and defending litigation); and* Archiving and record-keeping.LEGAL REMEDIES AND REGULATORY COMPLIANCEIn addition, we will use your information as permitted by local law, to protect our rights including to: (a) enforce this VECTOR Privacy Notice and our terms and conditions; (b) to protect our rights, privacy, safety, or property, and/or that of our affiliates, you or others; and (c) to allow us to pursue available remedies or limit the damages that we may sustain.Where required, we will use such information to:* comply with or evaluate compliance with any applicable law or regulation; and* respond to requests from public and government authorities, including public and government authorities outside your country of residence.CORPORATE TRANSACTIONSYour Personal Information may be used by us to complete a corporate transaction, such as a proposed or actual reorganization, merger, sale, joint venture, assignment, transfer or other disposition of all or any portion of our businesses, assets, or stock (including in connection with any bankruptcy or similar proceedings).Disclosure of Personal InformationWe may share Personal Information within The Walt Disney Family of Companies for the purposes described in this VECTOR Privacy Notice. We will make the information available to personnel with a need to know the information for the purposes described in this VECTOR Privacy Notice. In the course of our activities and for the same purposes as those listed in this VECTOR Privacy Notice, your Personal Information can be accessed by or transferred to the following categories of recipients on a need to know basis to achieve such purpose:* our banks and other payment processing service providers;* our IT systems providers, cloud service providers, database providers and consultants;* our other suppliers and service providers that provide services and products to us;* our professional service providers, such as accountants, auditors, legal advisors and insurance brokers; and* our archive storage providers; and* other third parties or entities as disclosed in this VECTOR Privacy Notice and permitted by law.Data RetentionWe will retain your Personal Information for the period necessary to fulfill the purposes outlined in this VECTOR Privacy Notice unless a longer retention period is required or permitted by applicable law.Access and CorrectionCertain websites and applications we provide may permit you to access, review, and change your Personal Information by logging in and updating your account information. Please note that if you change or correct information after submitting it, such changes will be visible to us.Apart from such information, you may request access to your Personal Information that we collect, or its modification, correction, or deletion, where permitted by applicable law. For your protection, we may only implement requests with respect to the information associated with the particular email address that you use to send us your request, and we may need to verify your identity before responding to your request. We will try to comply with your request as soon as reasonably practicable. Please note that certain Personal Information may be exempt from such access, correction, or suppression rights pursuant to local data protection laws.SecurityThe security, integrity, and confidentiality of your information are extremely important to us. We have implemented appropriate technical, administrative, organizational, and physical security measures that are designed to protect your information from unauthorized access, disclosure, use, and modification. We regularly review our security procedures to consider appropriate new technology and methods. Please be aware that, despite our best efforts, no security measures are perfect or impenetrable.Links to Third-Party WebsitesOur sites may contain links to other websites. This VECTOR Privacy Notice does not address, and we are not responsible for, the privacy, information security, or other practices of any third parties, including any third party operating any website or service to which our sites link. Privacy choices you have made on third party websites or services will not apply to our use of the information we have collected directly through our sites and offline. We encourage you to be aware when you leave our sites and to read the legal notices posted on other sites, including their privacy policies.Data Transfers and Applicable LawWe operate globally and may transfer your Personal Information to individual companies of The Walt Disney Family of Companies or third parties in locations around the world for the purposes described in this VECTOR Privacy Notice. Wherever your Personal Information is transferred, stored or processed by us, we will take reasonable steps to safeguard the privacy of your Personal Information. Additionally, when using or disclosing Personal Information transferred from the European Union, we use Standard Contractual Clauses approved by the European Commission, adopt other means under European Union law for ensuring adequate safeguards, or obtain your consent. We also apply the substantive requirements of standard contractual clauses when transferring Personal Information from Australia and Argentina.Changes to this VECTOR Privacy NoticeFrom time to time, we may change this VECTOR Privacy Notice to accommodate new technologies, industry practices, regulatory requirements or for other purposes. We will provide notice to you if these changes are material and, where required by applicable law, we will obtain your consent. Notice may be by email to you at the last email address you provided us, by posting notice of such changes on our sites and applications, or by other means, consistent with applicable law.Contact UsIf you have questions or requests related to this VECTOR Privacy Notice, please feel free to contact us by mail at: [email protected] If you are unsatisfied with a reply received from us, you may at any time refer your complaint to the relevant regulator in your jurisdiction.Additional Information for the EEA OnlyIf you are a resident of the UK or EEA, you may also:* Contact our Data Protection Officer at [email protected] with any questions about this VECTOR Privacy Notice.* Lodge a complaint with a data protection authority for your country or region or in the place of the alleged misconduct.DEFINITIONSData ControllerThe Data Controller is the subsidiary or affiliated entity of The Walt Disney Company that is responsible for the Personal Information collected and used within the scope of this VECTOR Privacy Notice from our sites and applications, as follows:* If you live in Europe, Middle East or Africa: The Walt Disney Company Limited (Company Number 530051) 3 Queen Caroline Street Hammersmith London W6 9PE [email protected]* US, Canada and Rest of World: Disney Interactive 500 South Buena Vista Street Mail Code 7667 Burbank, CA 91521-7667 United States of America Guest ServicesData ProcessorA data processor is a person or entity that processes Personal Information on behalf of a Data Controller (or Data Controllers) and is permitted to perform data processing only as directed by the Data Controller(s).Personal InformationPersonal Information means information that identifies, relates to, describes, is capable of being associated with, or could reasonably be linked (whether directly or indirectly) with a particular individual or household, such as the individual’s name, postal address, email address, and telephone number. When anonymous information is directly or indirectly associated with Personal Information, the resulting information also is treated as Personal Information.The Walt Disney Family of CompaniesThe Walt Disney Family of Companies refers to The Walt Disney Company and its subsidiary and affiliated entities, which offer their products and services under various brand names. These companies, including certain former 21st Century Fox assets, engage in a number of businesses, including theme parks and travel, motion pictures and television, publishing, consumer products and interactive services.I have read and understood the privacy policy.Submit

View More
Job Detail
Shortlist Never pay anyone for job application test or interview.