Skip to content

Validate Table Edm.DateTime writes and preserve legacy reads - #2870

Open
Justin Adler (PureKrome) wants to merge 2 commits into
Azure:mainfrom
PureKrome:fix/table-edm-datetime-range
Open

Justin Adler (PureKrome) wants to merge 2 commits into
Azure:mainfrom
PureKrome:fix/table-edm-datetime-range

Conversation

@PureKrome

Copy link
Copy Markdown

👋🏻 G'day Azurite team. I didn't make an issue because I don't have a question but found what i believe is a bug. I'm not sure if that is against the contributor's agreement (I did read it and could see anything that jumped out at me). If it's wrong, i can add an issue, etc.

This PR fixes Azurite denying "invalid" Edm.DateTime values so it's exactly like (production) Azure Table Storage. This prevents developers having code that works against Azurite and then explodes against ATS.

Table: reject Edm.DateTime values that Azure Table Storage doesn't support

📖 The rules (official Microsoft docs):

This follows the same approach as #2805, which fixed the same kind of problem for Edm.Int64 (#2558).

The problem

Azurite accepts any string as an Edm.DateTime, including dates before 1601 and text that isn't a date. Real Azure rejects these.

A common way to hit this: a .NET app leaves a DateTime property at its default value, DateTime.MinValue (0001-01-01T00:00:00.0000000Z). Tests pass against Azurite and then fail against Azure.

Before and after

Value sent as Edm.DateTime Why Before (3.37.0) After (this PR)
0001-01-01T00:00:00.0000000Z .NET DateTime.MinValue, too early ✅ 201 (wrong) ❌ 400 InvalidInput
1600-12-31T23:59:59.9999999Z 1 tick before the minimum ✅ 201 (wrong) ❌ 400 InvalidInput
1601-01-01T00:00:00.0000000Z Documented minimum ✅ 201 ✅ 201
9999-12-31T23:59:59.9999999Z Documented maximum ✅ 201 ✅ 201
not-a-date Not a date ✅ 201 (wrong) ❌ 400 InvalidInput
0001-01-01… via upsert (PUT) Too early ✅ 204 (wrong) ❌ 400 InvalidInput

What changed

  • src/table/entity/EdmDateTime.ts: validate(value, validateRange = true) now checks:
    • Format: the value must match one of the documented ISO 8601 forms and be a real calendar date. For example, 2023-02-29 and 24:00 are rejected. Otherwise it throws TypeError.
    • Range: after applying any time-zone offset, the value must be from 1601-01-01T00:00:00Z up to the end of 9999-12-31. Otherwise it throws RangeError.
  • EntityProperty.ts / NormalizedEntity.ts: the existing validateInt64Range flag from Validate Table Edm.Int64 writes and preserve legacy reads #2805 is renamed to validateRange and now applies to DateTime too.
  • Writes (insert, update, merge, upsert and batch, which all use the same handler code) already turn these errors into 400 InvalidInput.
  • Reads are not affected. TableHandler already reads with new NormalizedEntity(entity, false) (added in Validate Table Edm.Int64 writes and preserve legacy reads #2805). Out-of-range dates that older Azurite versions saved to a local database still read back fine.
  • Values without a time-zone suffix are still treated as UTC, as before.

Try it yourself

The script below has no dependencies and needs Node 18+. It sends raw REST requests using Azurite's well-known dev account. Save it as edm-datetime-repro.mjs.

edm-datetime-repro.mjs
// Sends Edm.DateTime values to Azurite's Table service and prints the HTTP status for each.
// No dependencies. Needs Node 18+ (uses the built-in fetch).
import { createHmac } from "node:crypto";

// Azurite's well-known development account. These are public, documented values,
// not secrets: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite
const account = "devstoreaccount1";
const key = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==";

// Azurite's default Table endpoint. Override with TABLE_ENDPOINT if yours differs.
const endpoint = process.env.TABLE_ENDPOINT ?? `http://127.0.0.1:10002/${account}`;

// Use a new table name on every run so repeated runs never clash.
const table = `edmdt${Date.now()}`;

// Sends one signed REST request and returns e.g. "201" or "400 InvalidInput".
async function call(method, path, body) {
  const url = new URL(`${endpoint}/${path}`);
  const date = new Date().toUTCString();

  // Table "SharedKeyLite" auth: HMAC-SHA256 over "<date>\n/<account><path>", signed with the account key.
  // This is the same signature the Azure SDKs compute; doing it by hand keeps the script dependency-free
  // and lets you see the exact JSON that goes over the wire.
  const signature = createHmac("sha256", Buffer.from(key, "base64"))
    .update(`${date}\n/${account}${decodeURIComponent(url.pathname)}`)
    .digest("base64");

  const res = await fetch(url, {
    method,
    headers: {
      "x-ms-date": date,
      "x-ms-version": "2019-02-02",
      Authorization: `SharedKeyLite ${account}:${signature}`,
      Accept: "application/json;odata=nometadata",
      "Content-Type": "application/json"
    },
    body: body && JSON.stringify(body)
  });

  // Error responses carry a code such as "InvalidInput" in the body; successful inserts have no error.
  const json = await res.json().catch(() => ({}));
  return `${res.status} ${json["odata.error"]?.code ?? ""}`.trim();
}

// 1. Create an empty table to insert into. Expect 201 (Created).
console.log("Create table:", await call("POST", "Tables", { TableName: table }));

// 2. The values to test. Azure's documented range is 1601-01-01 to 9999-12-31 (UTC).
const values = [
  "0001-01-01T00:00:00.0000000Z", // .NET DateTime.MinValue  -> should be rejected
  "1600-12-31T23:59:59.9999999Z", // 1 tick before minimum   -> should be rejected
  "1601-01-01T00:00:00.0000000Z", // documented minimum      -> OK
  "9999-12-31T23:59:59.9999999Z", // documented maximum      -> OK
  "not-a-date"                    // not a date at all       -> should be rejected
];

// 3. Insert one entity per value (POST = Insert Entity).
//    The "Value@odata.type" annotation is what tells the service to treat "Value" as an Edm.DateTime
//    rather than a plain string. Without it, any string would be accepted.
//    Expect 201 (Created) for valid values and 400 (Bad Request) for invalid ones.
for (const [i, value] of values.entries()) {
  const result = await call("POST", table, {
    PartitionKey: "p",
    RowKey: `row${i}`,
    Value: value,
    "Value@odata.type": "Edm.DateTime"
  });
  console.log(`POST ${value.padEnd(30)} -> ${result}`);
}

// 4. Try the too-early value again through a different write path: PUT = Insert Or Replace (upsert).
//    Expect 204 (No Content) if accepted, 400 if rejected.
const upsert = await call("PUT", `${table}(PartitionKey='p',RowKey='upsert')`, {
  Value: values[0],
  "Value@odata.type": "Edm.DateTime"
});
console.log(`PUT  ${values[0].padEnd(30)} -> ${upsert}`);

1. Before: the released Azurite (3.37.0) accepts the bad values

docker run -d --rm --name azurite-before -p 10002:10002 mcr.microsoft.com/azure-storage/azurite:3.37.0 azurite-table --tableHost 0.0.0.0
node edm-datetime-repro.mjs
docker stop azurite-before
Create table: 201
POST 0001-01-01T00:00:00.0000000Z   -> 201
POST 1600-12-31T23:59:59.9999999Z   -> 201
POST 1601-01-01T00:00:00.0000000Z   -> 201
POST 9999-12-31T23:59:59.9999999Z   -> 201
POST not-a-date                     -> 201
PUT  0001-01-01T00:00:00.0000000Z   -> 204

2. After: this branch rejects them

git fetch https://github.com/PureKrome/Azurite.git fix/table-edm-datetime-range
git checkout FETCH_HEAD
npm ci --legacy-peer-deps
npm run build
node dist/src/table/main.js --location ./tmp-azurite   # leave this running; run the script from a second terminal
node edm-datetime-repro.mjs
Create table: 201
POST 0001-01-01T00:00:00.0000000Z   -> 400 InvalidInput
POST 1600-12-31T23:59:59.9999999Z   -> 400 InvalidInput
POST 1601-01-01T00:00:00.0000000Z   -> 201
POST 9999-12-31T23:59:59.9999999Z   -> 201
POST not-a-date                     -> 400 InvalidInput
PUT  0001-01-01T00:00:00.0000000Z   -> 400 InvalidInput

3. Run the tests

npx mocha --require ts-node/register --no-timeouts --exit tests/table/unit/serialization.unit.test.ts
NODE_TLS_REJECT_UNAUTHORIZED=0 npx mocha --require ts-node/register --no-timeouts --exit tests/table/apis/table.validation.rest.test.ts

Tests added

  • tests/table/unit/serialization.unit.test.ts: boundary values, time-zone offsets, malformed strings, UTC handling, and reading old out-of-range data.
  • tests/table/apis/table.validation.rest.test.ts: insert and upsert return 400 InvalidInput; the min and max return 201; an old out-of-range value saved directly to the database still reads back through both GET paths.
  • Without the src changes, 4 of the new tests fail.
  • Locally (Node 22): npm run lint, npm run build, npm run test, npm run test:table and npm run test:table:in-memory all pass.

⚠️ One thing to check: the error code

This PR returns 400 InvalidInput for out-of-range dates, the same as #2805 does for Int64. I couldn't confirm the exact code real Azure returns today - I wasn't sure what to really use here. The only real error response I found is older (2012). It shows OutOfRangeInput ("One of the request inputs is out of range.") for DateTime.MinValue: http://www.anujchaudhary.com/2012/10/azure-table-storage-one-of-request.html. If a maintainer can confirm Azure still returns OutOfRangeInput, I'm happy to change it.

Checklist

  • PR targets main
  • Test cases added
  • ChangeLog.md updated under "Upcoming Release" › "Table:"

Reject Edm.DateTime property values outside the range Azure Table
Storage supports (1601-01-01T00:00:00Z to 9999-12-31T23:59:59.9999999Z)
or not in an accepted ISO 8601 format, returning 400 InvalidInput on
insert, update, merge, upsert and batch writes. Reads of out-of-range
values persisted by earlier Azurite versions keep working.

https://learn.microsoft.com/en-us/rest/api/storageservices/understanding-the-table-service-data-model#property-types
https://learn.microsoft.com/en-us/rest/api/storageservices/formatting-datetime-values

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 06:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Date-only normalization and strict legacy-read validation can make persisted entities unreadable.

2 open issues
What changed in this PR

Extends Table property validation to reject unsupported Edm.DateTime writes, following the existing Edm.Int64 approach.

Changes:

  • Adds format, calendar, and UTC range validation.
  • Shares the range-validation flag across numeric and date properties.
  • Adds unit and REST coverage plus release notes.
File Description
tests/​table/​unit/​serialization.unit.test.ts Tests date boundaries, formats, and legacy serialization.
tests/​table/​apis/​table.validation.rest.test.ts Tests write rejection and legacy reads.
src/​table/​entity/​NormalizedEntity.ts Generalizes the range-validation flag.
src/​table/​entity/​EntityProperty.ts Passes validation settings to DateTime properties.
src/​table/​entity/​EdmDateTime.ts Implements date format and range checks.
ChangeLog.md Documents the Table validation change.

Review effort: Balanced

Comment thread src/table/entity/EdmDateTime.ts Resolved
Comment thread src/table/entity/EdmDateTime.ts Resolved
Copilot AI balanced review requested due to automatic review settings October 21, 2026 06:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A maintainer should confirm Azure’s error code for out-of-range DateTime values before accepting API compatibility.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 21, 2026 06:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Azure’s exact error code for out-of-range DateTime writes needs confirmation before API compatibility can be approved.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants