--- title: "Hourly AWS Glue sync: about US$1,270 a month, and it never sees a deleted row" summary: "What the 2024 pipeline design on this blog costs at AWS's published Glue rates, why timestamp sync misses deletes, and what AWS changed, as of September 2026." url: https://www.thetechbag.com/blog/managing-versatile-data-pipeline-saas-cloud-platform format: Note desk: "Cloud & DevOps" published: 2026-06-09 facts_checked: 2026-09-30 author: TechBag (The TechBag Journal) tags: ["data pipeline","ETL","AWS Glue","Amazon Redshift","zero-ETL","AWS Data Pipeline","incremental sync","DPDP","erasure"] cover: https://www.thetechbag.com/blog/covers/managing-versatile-data-pipeline-saas-cloud-platform.png --- # Hourly AWS Glue sync: about US$1,270 a month, and it never sees a deleted row *What the 2024 pipeline design on this blog costs at AWS's published Glue rates, why timestamp sync misses deletes, and what AWS changed, as of September 2026.* Note · Cloud & DevOps · By TechBag · Published 2026-06-09 · Facts checked 2026-09-30 · Canonical: https://www.thetechbag.com/blog/managing-versatile-data-pipeline-saas-cloud-platform The pipeline this blog described in 2024, AWS Glue jobs copying changed rows into Redshift every hour, costs about US$1,270 a month per job at AWS’s list rate, and by design it never sees a row that was deleted. Our 2024 post reported that the design, run hourly at data-backup providers, moved 60 million records (45 GB) in 40 minutes. That is still a sound way to end data silos between microservices. Two things have changed since: what the law in India now asks of the copy in the warehouse, and what AWS itself now offers. ## What the hourly job costs AWS bills Glue’s Spark jobs at US$0.44 per DPU-hour, by the second. Its own pricing example uses a six-DPU job and prices the same job on Flex execution, meant for work that can wait for spare capacity, at a rate that works out to US$0.29.[1] **Figure: One hourly 40-minute Glue job lists at about US$1,267 a month; Flex cuts it by a third** *Monthly list cost in US$ of one six-DPU job running 40 minutes every hour, 720 runs, at AWS's published rates* Glue, standard execution US$0.44 per DPU-hour US$1,267 Glue, Flex execution US$0.29 per DPU-hour US$835 Aurora zero-ETL integration fee change I/O and Redshift compute still billed US$0 | | Value (US$) | | --- | --- | | Glue, standard execution | 1,267 | | Glue, Flex execution | 835 | | Aurora zero-ETL integration fee | 0 | Source: AWS Glue pricing page; TechBag arithmetic · as of 30 September 2026 · Redshift compute and storage are billed separately in every option TechBag countTechBag arithmetic Six DPUs for two-thirds of an hour at US$0.44 is US$1.76 a run; 24 runs a day for 30 days is 720 runs, or US$1,267.20. On Flex it is US$1.16 a run and US$835.20 a month. Six DPUs is AWS’s example size, not the 2024 post’s, which did not say; double the DPUs and the bill doubles.[1] ## The row it cannot see The 2024 design synced rows whose `created_timestamp` or `updated_timestamp` had moved since the last run. A deleted row has no timestamp to move, so the warehouse keeps it; the original post said as much, and pointed to soft-delete columns or triggers. India’s Digital Personal Data Protection Act, 2023 makes that gap a compliance question. Section 8(7) requires a data fiduciary to erase personal data, and have its processors erase it, once consent is withdrawn or the purpose is served; section 12 gives the person a right to ask.[2] Our view Our view: the 2024 post listed long-term retention in the warehouse as an advantage. Under the Act it is a liability to justify. A pipeline that cannot carry a delete into the warehouse leaves a copy the company cannot erase, so make deletes a tested path, with a soft-delete flag in every source table, before the warehouse holds customer data. ## What AWS changed One of the four AWS services our 2024 post named had already closed to new customers: AWS Data Pipeline stopped taking them on 25 July 2024, 48 days before that post, and AWS points migrating users to Glue, Step Functions or Managed Workflows for Apache Airflow.[3] It is in maintenance mode, with no new features planned.[4] For sources in Aurora, AWS now offers zero-ETL integration with Redshift at no additional fee, with data available “within seconds”; the change-data-capture I/O, the seed export and Redshift’s storage and compute are still billed.[5] Because it works from captured changes rather than timestamps, it is the first design to test for deletes: delete a test row and check the warehouse. **Buying it in India.** AWS publishes Glue in US dollars and says pricing can vary by Region, so price the job in the region the data sits in, Mumbai or Hyderabad.[1] An Indian invoice adds 18% GST. At ₹83 to the dollar, the standard-execution job above is about ₹1,05,178 a month, or ₹1,24,110 with GST. The DPDP Rules, notified on 13 November 2025, bring the erasure and security duties into force on 13 May 2027.[6] The Act’s schedule sets penalties of up to ₹250 crore for failing to keep reasonable security safeguards.[2] Our [databases and data tools guide](https://www.thetechbag.com/browse/devops/databases-data-tools) covers the engines these pipelines feed. How we did this Figures as of 30 September 2026 Glue rates and the six-DPU example are from AWS’s Glue pricing page (undated); the Flex rate is the one implied by AWS’s own Flex example. Monthly figures assume 720 runs (24 a day for 30 days), 40 minutes each, and exclude Redshift, Data Catalog, crawler and data transfer charges. The 60-million-record throughput is the 2024 post’s own report and was not re-measured. Zero-ETL terms are from AWS’s Aurora zero-ETL page (undated); its handling of deletes was not tested for this piece. Rupee figures convert at ₹83 to the dollar and add 18% GST; they are for scale, not a quote. Spotted an error? Write to us; corrections are published at the foot of the piece. Sources 1. [AWS, AWS Glue pricing (US$0.44 per DPU-hour; six-DPU and Flex examples; pricing can vary by Region)](https://aws.amazon.com/glue/pricing/) — undated 2. [Digital Personal Data Protection Act, 2023, sections 8(7) and 12, and Schedule (penalties)](https://www.meity.gov.in/data-protection-framework) — 11 August 2023 3. [AWS Big Data Blog, Migrate workloads from AWS Data Pipeline](https://aws.amazon.com/blogs/big-data/migrate-workloads-from-aws-data-pipeline/) — 25 July 2024 4. [AWS Data Pipeline Developer Guide, What is AWS Data Pipeline? (maintenance mode)](https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/what-is-datapipeline.html) — undated 5. [AWS, Amazon Aurora zero-ETL integration with Amazon Redshift (pricing and latency)](https://aws.amazon.com/rds/aurora/zero-etl/) — undated 6. [MeitY, Digital Personal Data Protection Rules, 2025 (G.S.R. 846(E))](https://www.meity.gov.in/static/uploads/2025/11/53450e6e5dc0bfa85ebd78686cadad39.pdf) — 13 November 2025 **Disclosure.** TechBag is a software reseller: it earns a margin when a reader buys through it, including from vendors named here. The analysis is TechBag’s own; no vendor paid for or saw this piece before publication. Facts are labelled reported, TechBag count or our view. Go deeper on TechBag - [Databases & data tools guide](https://www.thetechbag.com/browse/devops/databases-data-tools) — the engines and tooling a pipeline feeds - [Data security & privacy guides](https://www.thetechbag.com/browse/data-security-privacy) — finding and erasing personal data the DPDP Act covers