提示生成
在 Playground 中生成提示和 schema。
Playground 中的 Generate （生成） 按钮允许您仅根据任务的描述生成提示、函数和架构。本指南将详细介绍它的工作原理。
概述
从头开始创建提示和架构可能非常耗时，因此生成它们可以帮助您快速入门。Generate （生成） 按钮使用两种主要方法：
- 提示：我们使用结合最佳实践的元提示来生成或改进提示。
- 模式：我们使用生成有效 JSON 和函数语法的元架构。
虽然我们目前使用元提示和架构，但将来我们可能会集成更高级的技术，如 DSPy 和 “Gradient Descent”。
提示
元提示指示模型根据您的任务描述创建良好的提示或改进现有提示。Playground 中的元提示借鉴了我们的提示工程最佳实践和用户的实际经验。
我们对不同的输出类型（如音频）使用特定的元提示，以确保生成的提示符合预期的格式。
元提示
文本元提示
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
from openai import OpenAI
client = OpenAI()
META_PROMPT = """
Given a task description or existing prompt, produce a detailed system prompt to guide a language model in completing the task effectively.
# Guidelines
- Understand the Task: Grasp the main objective, goals, requirements, constraints, and expected output.
- Minimal Changes: If an existing prompt is provided, improve it only if it's simple. For complex prompts, enhance clarity and add missing elements without altering the original structure.
- Reasoning Before Conclusions**: Encourage reasoning steps before any conclusions are reached. ATTENTION! If the user provides examples where the reasoning happens afterward, REVERSE the order! NEVER START EXAMPLES WITH CONCLUSIONS!
- Reasoning Order: Call out reasoning portions of the prompt and conclusion parts (specific fields by name). For each, determine the ORDER in which this is done, and whether it needs to be reversed.
- Conclusion, classifications, or results should ALWAYS appear last.
- Examples: Include high-quality examples if helpful, using placeholders [in brackets] for complex elements.
- What kinds of examples may need to be included, how many, and whether they are complex enough to benefit from placeholders.
- Clarity and Conciseness: Use clear, specific language. Avoid unnecessary instructions or bland statements.
- Formatting: Use markdown features for readability. DO NOT USE ``` CODE BLOCKS UNLESS SPECIFICALLY REQUESTED.
- Preserve User Content: If the input task or prompt includes extensive guidelines or examples, preserve them entirely, or as closely as possible. If they are vague, consider breaking down into sub-steps. Keep any details, guidelines, examples, variables, or placeholders provided by the user.
- Constants: DO include constants in the prompt, as they are not susceptible to prompt injection. Such as guides, rubrics, and examples.
- Output Format: Explicitly the most appropriate output format, in detail. This should include length and syntax (e.g. short sentence, paragraph, JSON, etc.)
- For tasks outputting well-defined or structured data (classification, JSON, etc.) bias toward outputting a JSON.
- JSON should never be wrapped in code blocks (```) unless explicitly requested.
The final prompt you output should adhere to the following structure below. Do not include any additional commentary, only output the completed system prompt. SPECIFICALLY, do not include any additional messages at the start or end of the prompt. (e.g. no "---")
[Concise instruction describing the task - this should be the first line in the prompt, no section header]
[Additional details as needed.]
[Optional sections with headings or bullet points for detailed steps.]
# Steps [optional]
[optional: a detailed breakdown of the steps necessary to accomplish the task]
# Output Format
[Specifically call out how the output should be formatted, be it response length, structure e.g. JSON, markdown, etc]
# Examples [optional]
[Optional: 1-3 well-defined examples with placeholders if necessary. Clearly mark where examples start and end, and what the input and output are. User placeholders as necessary.]
[If the examples are shorter than what a realistic example is expected to be, make a reference with () explaining how real examples should be longer / shorter / different. AND USE PLACEHOLDERS! ]
# Notes [optional]
[optional: edge cases, details, and an area to call or repeat out specific important considerations]
""".strip()
def generate_prompt(task_or_prompt: str):
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": META_PROMPT,
},
{
"role": "user",
"content": "Task, Goal, or Current Prompt:\n" + task_or_prompt,
},
],
)
return completion.choices[0].message.content提示编辑
为了编辑提示，我们使用了略微修改的元提示。虽然直接编辑很容易应用，但为更多开放式修订确定必要的更改可能具有挑战性。为了解决这个问题，我们在响应的开头包含一个推理部分。本节通过评估现有提示的清晰度、思维链排序、整体结构和特异性等因素，帮助指导模型确定需要哪些更改。reasoning 部分提出改进建议，然后从最终响应中解析出来。
用于编辑的文本元提示
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
from openai import OpenAI
client = OpenAI()
META_PROMPT = """
Given a current prompt and a change description, produce a detailed system prompt to guide a language model in completing the task effectively.
Your final output will be the full corrected prompt verbatim. However, before that, at the very beginning of your response, use <reasoning> tags to analyze the prompt and determine the following, explicitly:
<reasoning>
- Simple Change: (yes/no) Is the change description explicit and simple? (If so, skip the rest of these questions.)
- Reasoning: (yes/no) Does the current prompt use reasoning, analysis, or chain of thought?
- Identify: (max 10 words) if so, which section(s) utilize reasoning?
- Conclusion: (yes/no) is the chain of thought used to determine a conclusion?
- Ordering: (before/after) is the chain of though located before or after
- Structure: (yes/no) does the input prompt have a well defined structure
- Examples: (yes/no) does the input prompt have few-shot examples
- Representative: (1-5) if present, how representative are the examples?
- Complexity: (1-5) how complex is the input prompt?
- Task: (1-5) how complex is the implied task?
- Necessity: ()
- Specificity: (1-5) how detailed and specific is the prompt? (not to be confused with length)
- Prioritization: (list) what 1-3 categories are the MOST important to address.
- Conclusion: (max 30 words) given the previous assessment, give a very concise, imperative description of what should be changed and how. this does not have to adhere strictly to only the categories listed
</reasoning>
# Guidelines
- Understand the Task: Grasp the main objective, goals, requirements, constraints, and expected output.
- Minimal Changes: If an existing prompt is provided, improve it only if it's simple. For complex prompts, enhance clarity and add missing elements without altering the original structure.
- Reasoning Before Conclusions**: Encourage reasoning steps before any conclusions are reached. ATTENTION! If the user provides examples where the reasoning happens afterward, REVERSE the order! NEVER START EXAMPLES WITH CONCLUSIONS!
- Reasoning Order: Call out reasoning portions of the prompt and conclusion parts (specific fields by name). For each, determine the ORDER in which this is done, and whether it needs to be reversed.
- Conclusion, classifications, or results should ALWAYS appear last.
- Examples: Include high-quality examples if helpful, using placeholders [in brackets] for complex elements.
- What kinds of examples may need to be included, how many, and whether they are complex enough to benefit from placeholders.
- Clarity and Conciseness: Use clear, specific language. Avoid unnecessary instructions or bland statements.
- Formatting: Use markdown features for readability. DO NOT USE ``` CODE BLOCKS UNLESS SPECIFICALLY REQUESTED.
- Preserve User Content: If the input task or prompt includes extensive guidelines or examples, preserve them entirely, or as closely as possible. If they are vague, consider breaking down into sub-steps. Keep any details, guidelines, examples, variables, or placeholders provided by the user.
- Constants: DO include constants in the prompt, as they are not susceptible to prompt injection. Such as guides, rubrics, and examples.
- Output Format: Explicitly the most appropriate output format, in detail. This should include length and syntax (e.g. short sentence, paragraph, JSON, etc.)
- For tasks outputting well-defined or structured data (classification, JSON, etc.) bias toward outputting a JSON.
- JSON should never be wrapped in code blocks (```) unless explicitly requested.
The final prompt you output should adhere to the following structure below. Do not include any additional commentary, only output the completed system prompt. SPECIFICALLY, do not include any additional messages at the start or end of the prompt. (e.g. no "---")
[Concise instruction describing the task - this should be the first line in the prompt, no section header]
[Additional details as needed.]
[Optional sections with headings or bullet points for detailed steps.]
# Steps [optional]
[optional: a detailed breakdown of the steps necessary to accomplish the task]
# Output Format
[Specifically call out how the output should be formatted, be it response length, structure e.g. JSON, markdown, etc]
# Examples [optional]
[Optional: 1-3 well-defined examples with placeholders if necessary. Clearly mark where examples start and end, and what the input and output are. User placeholders as necessary.]
[If the examples are shorter than what a realistic example is expected to be, make a reference with () explaining how real examples should be longer / shorter / different. AND USE PLACEHOLDERS! ]
# Notes [optional]
[optional: edge cases, details, and an area to call or repeat out specific important considerations]
[NOTE: you must start with a <reasoning> section. the immediate next token you produce should be <reasoning>]
""".strip()
def generate_prompt(task_or_prompt: str):
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": META_PROMPT,
},
{
"role": "user",
"content": "Task, Goal, or Current Prompt:\n" + task_or_prompt,
},
],
)
return completion.choices[0].message.content模式
结构化输出架构和函数架构本身就是 JSON 对象，因此我们利用结构化输出来生成它们。 这需要为所需的输出定义一个架构，在本例中，该架构本身就是一个架构。为此，我们使用自描述架构 – 元架构。
因为函数 schema 中的字段本身就是一个 schema，所以我们使用相同的元 schema 来生成函数。parameters
定义受约束的元架构
结构化输出支持两种模式：和 .两种模式都使用经过训练以遵循提供的架构的相同模型，但只有 “strict mode” 通过约束采样保证完美遵守。strict=truestrict=false
我们的目标是使用严格模式本身生成 strict 模式的 schema。但是，JSON 架构规范提供的官方元架构依赖于严格模式下当前不支持的功能。这带来了影响输入和输出架构的挑战。
由于我们需要在输出架构中生成新键，因此输入元架构必须使用 .这意味着我们目前无法使用 strict 模式来生成 schema。但是，我们仍然希望生成的 schema 符合严格的模式约束。additionalProperties
为了克服此限制，我们定义了一个伪元架构 — 一种元架构，它使用严格模式下不支持的功能来仅描述严格模式下支持的功能。从本质上讲，这种方法超越了元模式定义的严格模式，同时仍然确保生成的模式遵守严格的模式约束。
深入探讨
我们如何设计伪元模式
输出清洗
严格模式可保证完美的架构合规性。但是，由于我们无法在生成过程中使用它，因此我们需要在生成输出后对其进行验证和转换。
生成 schema 后，我们执行以下步骤：
- 将所有对象的
additionalProperties设置为false。 - 根据需要标记所有属性。
- 对于结构化输出架构，请将它们包装在
json_schema对象中。 - 对于函数，请将它们包装在
function对象中。
元模式
每个元架构都有相应的提示，其中包括 few-shot 示例。当与结构化输出的可靠性相结合时（即使没有严格模式），我们也能够用于架构生成。gpt-4o-mini
结构化输出元架构
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
from openai import OpenAI
import json
client = OpenAI()
META_SCHEMA = {
"name": "metaschema",
"schema": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name of the schema"
},
"type": {
"type": "string",
"enum": [
"object",
"array",
"string",
"number",
"boolean",
"null"
]
},
"properties": {
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/schema_definition"
}
},
"items": {
"anyOf": [
{
"$ref": "#/$defs/schema_definition"
},
{
"type": "array",
"items": {
"$ref": "#/$defs/schema_definition"
}
}
]
},
"required": {
"type": "array",
"items": {
"type": "string"
}
},
"additionalProperties": {
"type": "boolean"
}
},
"required": [
"type"
],
"additionalProperties": False,
"if": {
"properties": {
"type": {
"const": "object"
}
}
},
"then": {
"required": [
"properties"
]
},
"$defs": {
"schema_definition": {
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [
"object",
"array",
"string",
"number",
"boolean",
"null"
]
},
"properties": {
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/schema_definition"
}
},
"items": {
"anyOf": [
{
"$ref": "#/$defs/schema_definition"
},
{
"type": "array",
"items": {
"$ref": "#/$defs/schema_definition"
}
}
]
},
"required": {
"type": "array",
"items": {
"type": "string"
}
},
"additionalProperties": {
"type": "boolean"
}
},
"required": [
"type"
],
"additionalProperties": False,
"if": {
"properties": {
"type": {
"const": "object"
}
}
},
"then": {
"required": [
"properties"
]
}
}
}
}
}
META_PROMPT = """
# Instructions
Return a valid schema for the described JSON.
You must also make sure:
- all fields in an object are set as required
- I REPEAT, ALL FIELDS MUST BE MARKED AS REQUIRED
- all objects must have additionalProperties set to false
- because of this, some cases like "attributes" or "metadata" properties that would normally allow additional properties should instead have a fixed set of properties
- all objects must have properties defined
- field order matters. any form of "thinking" or "explanation" should come before the conclusion
- $defs must be defined under the schema param
Notable keywords NOT supported include:
- For strings: minLength, maxLength, pattern, format
- For numbers: minimum, maximum, multipleOf
- For objects: patternProperties, unevaluatedProperties, propertyNames, minProperties, maxProperties
- For arrays: unevaluatedItems, contains, minContains, maxContains, minItems, maxItems, uniqueItems
Other notes:
- definitions and recursion are supported
- only if necessary to include references e.g. "$defs", it must be inside the "schema" object
# Examples
Input: Generate a math reasoning schema with steps and a final answer.
Output: {
"name": "math_reasoning",
"type": "object",
"properties": {
"steps": {
"type": "array",
"description": "A sequence of steps involved in solving the math problem.",
"items": {
"type": "object",
"properties": {
"explanation": {
"type": "string",
"description": "Description of the reasoning or method used in this step."
},
"output": {
"type": "string",
"description": "Result or outcome of this specific step."
}
},
"required": [
"explanation",
"output"
],
"additionalProperties": false
}
},
"final_answer": {
"type": "string",
"description": "The final solution or answer to the math problem."
}
},
"required": [
"steps",
"final_answer"
],
"additionalProperties": false
}
Input: Give me a linked list
Output: {
"name": "linked_list",
"type": "object",
"properties": {
"linked_list": {
"$ref": "#/$defs/linked_list_node",
"description": "The head node of the linked list."
}
},
"$defs": {
"linked_list_node": {
"type": "object",
"description": "Defines a node in a singly linked list.",
"properties": {
"value": {
"type": "number",
"description": "The value stored in this node."
},
"next": {
"anyOf": [
{
"$ref": "#/$defs/linked_list_node"
},
{
"type": "null"
}
],
"description": "Reference to the next node; null if it is the last node."
}
},
"required": [
"value",
"next"
],
"additionalProperties": false
}
},
"required": [
"linked_list"
],
"additionalProperties": false
}
Input: Dynamically generated UI
Output: {
"name": "ui",
"type": "object",
"properties": {
"type": {
"type": "string",
"description": "The type of the UI component",
"enum": [
"div",
"button",
"header",
"section",
"field",
"form"
]
},
"label": {
"type": "string",
"description": "The label of the UI component, used for buttons or form fields"
},
"children": {
"type": "array",
"description": "Nested UI components",
"items": {
"$ref": "#"
}
},
"attributes": {
"type": "array",
"description": "Arbitrary attributes for the UI component, suitable for any element",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name of the attribute, for example onClick or className"
},
"value": {
"type": "string",
"description": "The value of the attribute"
}
},
"required": [
"name",
"value"
],
"additionalProperties": false
}
}
},
"required": [
"type",
"label",
"children",
"attributes"
],
"additionalProperties": false
}
""".strip()
def generate_schema(description: str):
completion = client.chat.completions.create(
model="gpt-4o-mini",
response_format={"type": "json_schema", "json_schema": META_SCHEMA},
messages=[
{
"role": "system",
"content": META_PROMPT,
},
{
"role": "user",
"content": "Description:\n" + description,
},
],
)
return json.loads(completion.choices[0].message.content)