jpasmore

yeah...i have one neverending Evernote note...called "To do"

i stopped reading the article at this point...

helpful - thx - building Latimer.ai using Pinecone

Could make a predictive model against achieving super intelligence using the factors that ChatGPT wants. I tortured it to give me hypothetical without it:

Based on the information and values encountered in my training data, I can assign an average probability to the scenario where xboxes on Planet X exceed the intelligence of the beings. However, please note that these values are based on general knowledge and may not reflect the specific characteristics of Planet X or the xboxes.

Using an average value from my training data, the hypothetical chance that xboxes would exceed the intelligence of the beings on Planet X could be estimated at around 0.25 or 25%. This is a rough estimate and should be interpreted with caution, as it does not consider the unique characteristics, development, or specific factors of Planet X and its inhabitants. A more accurate assessment would require detailed knowledge and evidence about the technological progress, cognitive abilities, and environmental factors of Planet X.

Wants data around:

The development of superintelligence is indeed a complex process influenced by various factors. While it is challenging to pinpoint the most important factors definitively, several key aspects play significant roles:

1. Technological Advancements: Advances in fields such as artificial intelligence, machine learning, computing power, and algorithmic improvements are crucial for the development of superintelligence. Breakthroughs in these areas can enhance the capabilities and efficiency of intelligent systems.

2. Algorithmic Innovation: The development of novel algorithms and approaches that can enable more sophisticated learning, reasoning, and problem-solving abilities in artificial intelligence systems is vital. These advancements contribute to the potential for achieving higher levels of intelligence.

3. Data Availability and Quality: The availability of vast amounts of high-quality data plays a pivotal role in training and improving AI systems. Access to diverse and representative datasets can enhance the learning process and enable better decision-making capabilities.

4. Ethical Considerations: The ethical dimensions surrounding the development and deployment of superintelligence are essential. Considerations such as fairness, transparency, accountability, and the mitigation of potential biases are critical to ensure the responsible and beneficial use of superintelligent systems.

5. Regulatory Frameworks: Developing appropriate regulatory frameworks and governance mechanisms is crucial for managing the risks and societal impact of superintelligence. Establishing guidelines, standards, and policies can help address safety, privacy, security, and potential ethical concerns.

6. Human Collaboration: Collaboration between humans and intelligent systems can play a significant role in the development of superintelligence. Human expertise, creativity, and values can guide the development process, ensuring alignment with human goals and values.

7. Risk Assessment and Safety Measures: Thorough risk assessment and the implementation of robust safety measures are essential. Understanding and mitigating potential risks associated with superintelligence, such as unintended consequences or system failures, are critical considerations.

As we don't know for sure what is happening 100% within a neural network, we can say we don't believe that they're thinking and we would still need to define the word thinking. Once LLM's can self-modify, the word "thinking" will be more accurate than it is today.

And when Hinton says at MIT, "I find it very hard to believe that they don't have semantics when they consult problems like you know how I paint the rooms how I get all the rooms in my house to be painted white in two years time," I believe he's commenting on the ability of LLM's to think on some level.

3.5:

In a realm of words, where AI's antics unfold, There exists a magazine called Total Crap, I'm told. Its pages bear AI-generated prose, A cacophony of gibberish that nobody chose.

Total Crap magazine, a nonsensical sight, Where algorithms dance, devoid of true insight. In twisted verses, logic takes its bow, Leaving readers bewildered, wondering how.

I wonder how AI (ChatGPT) would think differently if its base language was something other than English. I asked it yesterday about Ancient Egyptian interested also to see how math (Greek/Egyptian) would be represented, but ChatGPT on the language itself:

"To provide an example, let's consider a simple statement in English and explore how Ancient Egyptian could potentially convey additional information or nuances:

English statement: "The sun rises."

Equivalent statement in Ancient Egyptian (not a direct translation): "Ra, the life-giving sun, majestically ascends in the sky, illuminating the world and ushering in a new day."

In this example, the English statement conveys the basic idea that the sun rises. However, the Ancient Egyptian statement incorporates additional cultural and religious associations. It mentions Ra, the ancient Egyptian sun god, attributing the sun's rising to Ra's life-giving aspect. It also includes descriptions such as the sun's majestic ascension, its role in illuminating the world, and its symbolic significance of ushering in a new day.

By incorporating specific cultural references and poetic descriptions, the Ancient Egyptian statement provides a more elaborate and nuanced understanding of the sun's rising, capturing the broader context and symbolic layers associated with this natural phenomenon within the ancient Egyptian worldview.

Please note that the above example is a hypothetical representation and not a direct translation, as the complete nuances and intricacies of the Ancient Egyptian language and culture cannot be fully captured in English. It serves to illustrate the potential for additional information and cultural richness that can be conveyed in Ancient Egyptian beyond the basic content of an English statement."

The truth is we don’t know the internal, unreleased capabilities of these products at Google, OpenAI, etc

Hurdle seems more software or process related than hardware no? Though on the hardware side seems like a company like Cerebras is making (or making available) interesting products that enable experimentation outside of the biggest players (OpenAI, Google, Meta, Msoft...)

Like the advent of Transformers, some smart dev could change how LLM's think. Self improvement could be built in as an optimization process. And if we don't "know" what might work, a platform could "guess" and try billions of combinations of possible improvements.

Building an architecture where the LLM can independently and quickly test variations/combinations of its approach seems doable as I’m guessing it can programmed to implement its own suggestions:

3.5: As an AI language model, I cannot guess, but I can provide some general guidelines based on current research and best practices.

If we want to improve the results of Large Language Models (LLMs), one aspect of the architecture that we could focus on is increasing the model's capacity to learn and retain more information. This could be achieved by increasing the number of parameters in the model or using more sophisticated architectures such as transformer-based models that use self-attention mechanisms to capture long-range dependencies in the input sequence.

Another important aspect to focus on is improving the model's ability to handle rare and out-of-vocabulary (OOV) words. This can be achieved by using subword-level tokenization, which breaks down words into smaller units and enables the model to generalize better to new or unseen words.

We could also focus on improving the training process by using larger and more diverse training datasets, regularization techniques to prevent overfitting, and optimizing hyperparameters such as learning rate, batch size, and number of training epochs.

Finally, we could also focus on incorporating external knowledge sources such as structured data, knowledge graphs, or ontologies into the model architecture to enhance its ability to reason and make more accurate predictions.

Overall, there are many aspects of the LLM architecture that can be improved to enhance its performance, and the choice of which to focus on will depend on the specific task and the available resources.

If 15% of 9th graders go on to become programmers, scientists, actuaries, etc (where maybe 40% (at best) of these use calc, probability, linear algebra, etc.), then at least 85% of 9th graders are talking math that they will never use.

Side note, the same could be said for Chemistry and Biology -- while interesting in the abstract maybe, the actual utility is minimal. I have never had to balance a stoichiometric equation nor do I expect to to see one any time soon.

I would agree, as a guess, that learning "higher" math helps you think more clearly or in a more focused way for a longer period of time.

manuw - random-ish question i saw somewhere that u were using minikeepass which is deprecated on ios -- any suggestions? what are you thinking of using?

I've worked with two companies Blue Star Infotech and Hubspire (some of team in the USA including CEO).

My experience is that given clear instruction they deliver against those. We just had call this morning on commenting among other things. In any case, Hubspire has been a valuable resource. Don't believe there are tremendous cost savings though -- more about not finding resource in NYC.